common-query
Advanced tools
A javascript implementation of the MongoDB-style query and update syntax. It also has some useful methods built on top, and provides an extensible framework for adding new query and update operators.
let commonQuery = require('common-query');
let createQuery = commonQuery.createQuery;
let createUpdate = commonQuery.createUpdate;
let obj = { foo: 'bar', abc: 3 };
let query = createQuery({
$and: [ {
foo: 'bar'
}, {
abc: {
$in: [ 1, 2, 3 ]
}
} ]
});
console.log(query.matches(obj)); // true
let update = createUpdate({
$set: {
foo: 'baz'
},
$inc: {
abc: 2
}
});
let newObj = update.apply(obj);
console.log(newObj); // { foo: 'baz', abc: 5 }
console.log(query.matches(newObj)); // false
Note that createQuery(), createUpdate, query.matches(), and query.apply() can all throw errors if the query/update data object is invalid, or if the query/update is applied to an invalid object.
The following mongo query operators are supported by default:
$and$or$nor$exists$not$elemMatch$in$nin$all$size$text$regex$gt$gte$lt$lte$neSee the MongoDB docs here for documentation on these operators.
The following extra operators are included:
$wildcard: A simplified version of $regex, supporting glob-like expressions using the operators * and ?.$var: See the detailed query documentation below. (Not really an operator)The following mongo update operators are supported by default:
$set$unset$inc$mul$rename$min$max$addToSet$push$popSee the MongoDB docs here for documentation on these operators.
createQuery() takes an optional options object as the second argument - see the docs directory for a complete list.
A query with variable parameters can be specified using the 'fake' $var operator. Example:
let queryData = {
name: 'Henrietta Wilkinson',
age: 21,
favoriteColor: { $var: 'color' }
};
let query = createQuery(queryData, {
vars: {
color: 'green'
}
});
console.log(query.matches({
age: 21,
favoriteColor: 'green'
})); // true
During query construction, the $var objects are replaced with the value specified by the vars option. Note that
any missing $var substitution will result in an invalid query.
The query object also includes some additional functionality:
// Get the plain object that represents this query
query.getData();
// Get the query factory used to generate this query (see below)
query.getQueryFactory();
/* Get a generated function to match a query. This may be more efficient than query.matches(), and should
be used if a query is to be used many times. */
let func = query.createMatchFn();
func(objectToMatch); // true
/* Normalize a query, optionally to a provided schema. Validates the query as well.
This is done by default in the constructor; pass the skipValidate option to opt out of
this behavior. */
query.normalize();
query.normalize({
schema: createSchema({
foo: String,
bar: [ { baz: [ { qux: Number } ] } ]
})
});
/* Ensure that a query is valid; will throw an error if the query is invalid. This is done by default
in the constructor; pass the skipValidate option to opt out of this behavior. Note that calling
matches() on an invalid query results in undefined behavior. */
query.validate(); // throws QueryValidationError
// Get a list of fields that a query will access during matching
query.getQueriedFields(); // [ 'field1', 'field2', 'field3' ]
// Get a list of fields that a query matches exactly. These are fields that must match a single scalar value.
// See the code documentation for details.
query.getExactMatches(); // { exactMatches: [ 'field1', 'field2' ], onlyExactMatches: false }
// Get a list of operators used by this query
query.getOperators(); // [ '$and', '$gt', '$regex' ]
createUpdate() takes an optional options object as the second argument. Some options are:
$set
operation, and updated only the stated fields. If you set allowFullReplace, this will not occur, and such
an update will replace the entire object (as is the default MongoDB behavior).See the docs directory for a complete list.
On top of apply(), the update object also includes some additional functionality:
// Get the plain object that represents this update
update.getData();
// Get the update factory used to generate this update (see below)
update.getUpdateFactory();
/* Get a generated function to apply an update. This may be more efficient than update.apply(),
and should be used if a query is to be used many times. */
let func = update.createUpdateFn();
func(objectToUpdate); // true
/* Normalize an update, optionally to a provided schema. Validates the update as well.
This is done by default in the constructor; pass the skipValidate option to opt out of
this behavior. */
update.normalize();
update.normalize({
schema: createSchema({
foo: String,
bar: [ { baz: [ { qux: Number } ] } ]
})
});
/* Ensure that an update is valid; will throw an error if the update is invalid. This is done by default
in the constructor; pass the skipValidate option to opt out of this behavior. Note that calling
apply() on an invalid update results in undefined behavior. */
update.validate(); // throws UpdateValidationError
// Get a list of fields that will be updated
update.getUpdatedFields(); // [ 'field1', 'field2', 'field3' ]
// Get a list of operators used by this update
update.getOperators(); // [ '$set', '$inc', '$addToSet' ]
// Whether or not the field contains update operators
update.hasOperators(); // true
// Whether the update has no operators, and will thus be a full object replacement
// (this is the inverse of update.hasOperators())
update.isFullReplace(); // false
In addition to the instance methods, there are also some notable static methods on the Update class.
// Create an update from the recursive diff of two objects
// See the method's documentation for accepted options
Update.createFromDiff({
foo: 'bar',
baz: true,
plork: [ { asdf: true }, { aoeu: true } ]
}, {
foo: 'boar',
boom: 4,
plork: [ { aoeu: true, asdf: false } ]
});
// Results in:
{
"$set": {
"foo": "boar",
"boom": 4,
"plork.0.aoeu": true,
"plork.0.asdf": false
},
"$unset": {
"baz": true,
"plork.1": true
},
"$push": {
"plork": {
"$slice": 1
}
}
}
Each query and update is created from an QueryFactory or UpdateFactory, respectively. The createQuery() and createUpdate() functions are shorthand for getting these entities from a default factory. A more complete, equivalent way to create a query or update would be:
let queryFactory = new commonQuery.QueryFactory();
let query = queryFactory.createQuery(/* queryData */, /* options */);
let updateFactory = new commonQuery.UpdateFactory();
let update = updateFactory.createUpdate(/* updateData */, /* options */);
Each factory will load the default set of operators when it is instantiated. Additional custom operators can be added by subclassing QueryOperator or ExprOperator for queries, or UpdateOperator for updates, and registering them to the query factory. An example follows:
class HorseUpdateOperator extends commonQuery.ExprOperator {
constructor(name) {
super(name || '$horse');
}
matchesValue(value) {
return ([ 'horse', 'foal', 'colt', 'pony' ].indexOf(value) !== -1);
}
}
let queryFactory = new commonQuery.QueryFactory();
queryFactory.registerExprOperator('$horse', new HorseUpdateOperator());
let query = queryFactory.createQuery({
name: {
$in: [ 'Shirley', 'Emma', 'Carly Rae Jepsen' ]
},
favoriteAnimal: {
$horse: true
}
});
console.log(query.matches({
name: 'Emma',
favoriteAnimal: 'pony'
})); // true
See the docs directory for full documentation on creating and registering custom operators.
Unimodel aggregates are specified in a common format that has no analog (that I know of) in existing systems. The best way to understand them is by example. Examples in this section use documents representing pets at animal shelters:
{
animalType: 'dog',
animalSubtype: 'Yorkshire Terrier',
age: 4,
weight: 14.2,
shelterLocation: 'Clifton',
dateFound: '2013-03-20T04:13:23Z',
name: 'Ruff'
}
This aggregate type returns statistics on a field across a whole collection (or a subset matched by a query).
{
// Perform statistics on a field
stats: {
age: {
count: true,
avg: true,
max: true
}
},
// Return the total number of documents the aggregate is executed across
total: true
}
A result set for this aggregate would look something like:
{
// There are 400 animals matched by the query
total: 400,
stats: {
age: {
// Of those, 329 have non-null 'age' fields
count: 329,
// The average age of animals in 5.2382
avg: 5.2382,
// The maximum age of animals is 19.2
max: 19.2
}
}
}
The different types of stats you can ask for are:
Not all model types need support all of these types of stats, and model types may add additional stats if they are supported.
The stats object is a mapping between field paths and the statistics to perform. You can also supply more than one stats field in the aggregate:
{
stats: {
age: {
max: true
},
dateFound: {
min: true
}
}
}
Results might look like this:
{
stats: {
age: {
max: 19.2
},
dateFound: {
min: '2015-04-12T07:22:09Z'
}
}
}
For convenience, stats can be a single string. In this case, the string is treated as
a field name, and the count stat is executed on it:
{
stats: 'animalType'
}
is converted to:
{
stats: {
animalType: {
count: true
}
}
}
This type of aggregate will return statistics grouped by discrete values of a field.
{
// The field to group by is 'animalType'
groupBy: [ {
field: 'animalType'
} ],
// Perform statistics within each group on the 'age' field
stats: {
age: {
avg: true
}
},
// Return the total number of documents in each group
total: true
}
Results look like:
[
{
// The value of the groupBy field (see below for why this is an array)
key: [ 'cat' ],
// Requested statistics for this grouping
stats: {
age: {
// Average age of cats
avg: 7.2
}
},
// There are 18 cats in the database (note that this is outside the field stats blocks)
total: 18
},
{
key: [ 'dog' ],
stats: {
age: {
avg: 6.4
}
},
total: 12
},
{
key: [ 'bird' ],
stats: {
age: {
avg: 2.1
}
},
total: 4
}
]
As a shorthand, you can specify the groupBy field as a single object:
{
groupBy: {
field: 'animalType'
}
}
or as a string:
{
groupBy: 'animalType'
}
Both shorthand forms are converted to:
{
groupBy: [ {
field: 'animalType'
} ]
}
You can also leave off stats to get only totals:
{
groupBy: [ { field: 'animalType' } ],
total: true
}
May yield:
[
{
key: [ 'cat' ],
total: 18
},
{
key: [ 'dog' ],
total: 12
},
{
key: [ 'bird' ],
total: 4
}
]
If the groupBy field is an array, each of the array's elements is treated as a separate
entry. This means that, in this case, the counts and totals in the aggregate result won't
equal the total number of documents (documents can be counted multiple times if the array
has multiple elements).
This will group by ranges of a numeric or date field.
{
groupBy: [ {
// Numeric/date field to group by
field: 'age',
ranges: [
// First group (group 0) is animals less than 1 year old
{ end: 1 },
// Second group (group 1) is animals 1-3 years old
{ start: 1, end: 3 },
// Third group (group 2) is animals 3-9 years old
{ start: 3, end: 9 },
// Fourth group (group 3) is animals more than 9 years old
{ start: 9 }
]
} ],
// Give total matching for each group
// Note that you can also supply stats here as well
total: true
}
Results look like this:
[
{
// This is the entry for group number 0
// These indices correspond to the indices in the given ranges array
key: [ 0 ],
// There are 5 animals in this range (less than 1 year old)
total: 5
},
{
key: [ 1 ],
total: 8
},
{
key: [ 2 ],
total: 14
},
{
key [ 3 ],
total: 7
}
]
These ranges can also be dates if applied to a date field:
{
groupBy: [ {
field: 'dateFound',
ranges: [
{ end: '2010-01-01T00:00:00Z' },
{ start: '2010-01-01T00:00:00Z', end: '2013-01-01T00:00:00Z' },
{ start: '2013-01-01T00:00:00Z' }
]
} ],
total: true
}
For convenience, a continuous series of non-overlapping ranges can be specified as:
{
groupBy: [ {
field: 'age',
ranges: [ 1, 3, 9 ]
} ],
total: true
}
Will be converted to:
{
groupBy: [ {
field: 'age',
ranges: [
{ end: 1 },
{ start: 1, end: 3 },
{ start: 3, end: 9 },
{ start: 9 }
]
} ],
total: true
}
The output of this is:
[
{
// This key corresponds to the range ENDING at index 0 (ie, all animals less than 1 year old)
key: [ 0 ],
// There are 5 animals in this range (less than 1 year old)
total: 5
},
{
// Range from 1-3 years
key: [ 1 ],
total: 8
},
{
// Range from 3-9 years
key: [ 2 ],
total: 14
},
{
// One more result entry than entries in the array
// This is for animals more than 9 years old
key [ 3 ],
total: 7
}
]
Strings found in start/end properties will also attempt to be parsed into number/date values. For example:
{
groupBy: [ {
field: 'age',
ranges: [ '3', 5 ]
} ],
total: true
}
will be converted to:
{
groupBy: [ {
field: 'age',
ranges: [
{ end: 3 },
{ start: 3, end: 5 },
{ start: 5 }
]
} ],
total: true
}
and:
{
groupBy: [ {
field: 'dateFound',
ranges: [ '2015-01-01T05:00:00.000Z', new Date(2015, 1, 1, 0, 0, 0) ]
} ],
total: true
}
will be converted to:
{
groupBy: [ {
field: 'age',
ranges: [
{ end: new Date('2015-01-01T00:00:00.000Z') },
{ start: new Date('2015-01-01T00:00:00.000Z'), end: new Date('2015-02-01T00:00:00.000Z') },
{ start: new Date('2015-02-01T00:00:00.000Z') }
]
} ],
total: true
}
This will group continuous values across fixed intervals.
{
groupBy: [ {
// Segment the numeric field 'age'
field: 'age',
// Each interval is of length 3
interval: 3,
// By default, intervals start at 0 (ie, -3, 0, 3, 6, 9, etc)
// This supplies a different offset
// When set to 1, the intervals become -2, 1, 4, 7, etc
base: 1
} ],
total: true
}
Results look like this:
[
{
// The key here is the start value of the interval
// Ie, this entry is for the interval -2 through 1
key: [ -2 ],
total: 5
},
{
// This is for the interval 1 through 4
key: [ 1 ],
total: 4
},
{
key: [ 4 ],
total: 8
},
...
]
These can also be applied to dates. In this case, the interval should be supplied as an ISO 8601 time Duration. For example, an interval of 'P3H15M' is an interval of 15 minutes.
{
groupBy: [ {
field: 'dateFound',
interval: 'P8H',
// The default base when using time intervals is not defined.
// Override bases are specified as an ISO8601 timestamp.
base: '2010-01-01T00:00:00Z'
} ],
total: true
}
Results in:
[
{
// Result keys are ISO timestamps
key: [ '2010-01-01T00:00:00Z' ],
total: 2
},
{
key: [ '2010-01-01T08:00:00Z' ],
total: 1
},
...
]
As a convenience, you string values from base and interval will be converted to proper number/date values. For instance:
{
groupBy: [ {
field: 'age',
interval: '3',
base: '1'
} ],
total: true
}
will be converted to:
{
groupBy: [ {
field: 'age',
interval: 3,
base: 1
} ],
total: true
}
Usually, when you want to group by (for example) month, you don't actually want to use a time interval of 30 days because these won't align with month boundaries. This grouping type allows you to group by time components.
{
groupBy: [ {
// Field to group by
field: 'dateFound',
// Time component to group into
timeComponent: 'year',
// The number of time components in each group (optional)
timeComponentCount: 1
} ],
total: true
}
The output looks like:
[
{
key: [ '2012-01-01T00:00:00Z' ],
total: 4
},
{
key: [ '2013-01-01T00:00:00Z' ],
total: 7
},
{
key: [ '2014-01-01T00:00:00Z' ],
total: 5
},
...
]
Each of the result keys is an ISO8601 timestamp corresponding to the start of the range represented by that time component.
The timeComponent field can be one of the following:
yearmonthweekdayhourminutesecondThe timeComponentCount field is optional, and can be used to create longer intervals.
{
type: 'group',
groupBy: [ {
field: 'dateFound',
timeComponent: 'day',
timeComponentCount: 2
} ],
total: true
}
Can result in:
[
{
key: [ '2012-01-01T00:00:00Z' ],
total: 1
},
{
key: [ '2012-01-03T00:00:00Z' ],
total: 2
},
{
key: [ '2012-01-05T00:00:00Z' ],
total: 2
},
...
]
Note that this does NOT represent a duration. The last interval in the range of a time component may be cut short (for example, in months with 31 days, the last interval in the above example would be only a single day instead of 2 days).
The "base" value for a time component is always the first valid point in time for that component.
For year, the base point in time used is year 1.
The groupBy parameter in it's most verbose form is an array of goruping specifiers.
When multiple specifiers are present, a powerset of the results will be producsed.
For example:
{
groupBy: [
{
field: 'animalType'
},
{
field: 'age',
interval: 4
}
],
total: true
}
This groups by age (in intervals of 4) and animalType. The results for this look like:
[
{
key: [ 'dog', 0 ],
total: 2
},
{
key: [ 'dog', 4 ],
total: 3
},
{
key: [ 'dog', 8 ],
total: 2
},
{
key: [ 'cat', 0 ],
total: 5
},
{
key: [ 'cat', 4 ],
total: 3
},
...
]
You may specify a whitelist of keys to include in the result for each groupBy entry:
{
groupBy: {
field: 'animalType',
only: [ 'cat', 'dog' ]
},
total: true
}
May yield:
[
{
key: [ 'cat' ],
total: 18
},
{
key: [ 'dog' ],
total: 12
}
]
FAQs
Base framework for common mongo-style query language
We found that common-query demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Following last week’s supply chain attack, Nx published findings on the GitHub Actions exploit and moved npm publishing to Trusted Publishers.
Security News
AGENTS.md is a fast-growing open format giving AI coding agents a shared, predictable way to understand project setup, style, and workflows.
Security News
/Research
Malicious npm package impersonates Nodemailer and drains wallets by hijacking crypto transactions across multiple blockchains.