sift

What is sift?

The sift npm package is a library that provides a way to use MongoDB-like query syntax for filtering, sorting, and manipulating JavaScript arrays and objects. It is commonly used for client-side querying of JSON data, in-memory database emulation, and other applications where MongoDB query semantics are desired without a database.

What are sift's main functionalities?

Filtering Arrays

This code sample demonstrates how to filter an array of objects to find elements where the age is greater than 18 and the name starts with the letter 'J'.

{"$and":[{"age":{"$gt":18}},{"name":{"$regex":"^J"}}]}

Sorting Arrays

This code sample shows how to sort an array of objects by the age property in ascending order.

{"$sort":{"age":1}}

Projection

This code sample illustrates how to create a projection of an array of objects, including only the name and age properties in the output.

{"$project":{"name":1,"age":1}}

Other packages similar to sift

mongodb

The official MongoDB driver for Node.js. It allows you to work with MongoDB databases directly. While it provides similar querying capabilities, it is designed for server-side use with an actual MongoDB database rather than in-memory operations like sift.

mongoose

An Object Data Modeling (ODM) library for MongoDB and Node.js. Mongoose provides a straight-forward, schema-based solution to model application data. It includes built-in type casting, validation, query building, and business logic hooks. It is more complex and feature-rich compared to sift, and it is intended for use with MongoDB databases.

lodash

A modern JavaScript utility library delivering modularity, performance, & extras. Lodash provides a wide range of functions for manipulating and querying data, but it does not use MongoDB-like query syntax. It is more general-purpose compared to sift.

underscore

Underscore is a utility-belt library for JavaScript that provides functional programming helpers without extending any built-in objects. It's similar to lodash and does not offer MongoDB-like query syntax, but it can be used for data manipulation and querying.

README

Installation: npm install sift, or yarn add sift

Sift is a tiny library for using MongoDB queries in Javascript

For extended documentation, checkout http://docs.mongodb.org/manual/reference/operator/query/

Features:

• Supported operators: $in, $nin, $exists, $gte, $gt, $lte, $lt, $eq, $ne, $mod, $all, $and, $or, $nor, $not, $size, $type, $regex, $where, $elemMatch
• Regexp searches
• Supports node.js, and web
• Custom Operations
• Tree-shaking (omitting functionality from web app bundles)

Examples

import sift from "sift";

// intersecting arrays
const result1 = ["hello", "sifted", "array!"].filter(
  sift({ $in: ["hello", "world"] }),
); // ['hello']

// regexp filter
const result2 = ["craig", "john", "jake"].filter(sift(/^j/)); //['john','jake']

// function filter
const testFilter = sift({
  //you can also filter against functions
  name: function (value) {
    return value.length == 5;
  },
});

const result3 = [
  {
    name: "craig",
  },
  {
    name: "john",
  },
  {
    name: "jake",
  },
].filter(testFilter); // filtered: [{ name: 'craig' }]

// you can test *single values* against your custom sifter
testFilter({ name: "sarah" }); //true
testFilter({ name: "tim" }); //false

API

sift(query: MongoQuery, options?: Options): Function

Creates a filter with all the built-in MongoDB query operations.

• query - the filter to use against the target array
• options
  • operations - custom operations
  • compare - compares difference between two values

Example:

import sift from "sift";

const test = sift({ $gt: 5 });

console.log(test(6)); // true
console.log(test(4)); // false

[3, 4, 5, 6, 7].filter(test); // [6, 7]

createQueryTester(query: Query, options?: Options): Function

Creates a filter function without built-in MongoDB query operations. This is useful if you're looking to omit certain operations from application bundles. See Omitting built-in operations for more info.

import { createQueryTester, $eq, $in } from "sift";
const filter = createQueryTester({ $eq: 5 }, { operations: { $eq, $in } });

createEqualsOperation(params: any, ownerQuery: Query, options: Options): Operation

Used for custom operations.

import { createQueryTester, createEqualsOperation, $eq, $in } from "sift";
const filter = createQueryTester(
  { $mod: 5 },
  {
    operations: {
      $something(mod, ownerQuery, options) {
        return createEqualsOperation(
          (value) => value % mod === 0,
          ownerQuery,
          options,
        );
      },
    },
  },
);
filter(10); // true
filter(11); // false

Supported Operators

See MongoDB's advanced queries for more info.

$in

array value must be $in the given query:

Intersecting two arrays:

// filtered: ['Brazil']
["Brazil", "Haiti", "Peru", "Chile"].filter(
  sift({ $in: ["Costa Rica", "Brazil"] }),
);

Here's another example. This acts more like the $or operator:

[{ name: "Craig", location: "Brazil" }].filter(
  sift({ location: { $in: ["Costa Rica", "Brazil"] } }),
);

$nin

Opposite of $in:

// filtered: ['Haiti','Peru','Chile']
["Brazil", "Haiti", "Peru", "Chile"].filter(
  sift({ $nin: ["Costa Rica", "Brazil"] }),
);

$exists

Checks if whether a value exists:

// filtered: ['Craig','Tim']
sift({ $exists: true })(["Craig", null, "Tim"]);

You can also filter out values that don't exist

// filtered: [{ name: "Tim" }]
[{ name: "Craig", city: "Minneapolis" }, { name: "Tim" }].filter(
  sift({ city: { $exists: false } }),
);

$gte

Checks if a number is >= value:

// filtered: [2, 3]
[0, 1, 2, 3].filter(sift({ $gte: 2 }));

$gt

Checks if a number is > value:

// filtered: [3]
[0, 1, 2, 3].filter(sift({ $gt: 2 }));

$lte

Checks if a number is <= value.

// filtered: [0, 1, 2]
[0, 1, 2, 3].filter(sift({ $lte: 2 }));

$lt

Checks if number is < value.

// filtered: [0, 1]
[0, 1, 2, 3].filter(sift({ $lt: 2 }));

$eq

Checks if query === value. Note that $eq can be omitted. For $eq, and $ne

// filtered: [{ state: 'MN' }]
[{ state: "MN" }, { state: "CA" }, { state: "WI" }].filter(
  sift({ state: { $eq: "MN" } }),
);

Or:

// filtered: [{ state: 'MN' }]
[{ state: "MN" }, { state: "CA" }, { state: "WI" }].filter(
  sift({ state: "MN" }),
);

$ne

Checks if query !== value.

// filtered: [{ state: 'CA' }, { state: 'WI'}]
[{ state: "MN" }, { state: "CA" }, { state: "WI" }].filter(
  sift({ state: { $ne: "MN" } }),
);

$mod

Modulus:

// filtered: [300, 600]
[100, 200, 300, 400, 500, 600].filter(sift({ $mod: [3, 0] }));

$all

values must match everything in array:

// filtered: [ { tags: ['books','programming','travel' ]} ]
[
  { tags: ["books", "programming", "travel"] },
  { tags: ["travel", "cooking"] },
].filter(sift({ tags: { $all: ["books", "programming"] } }));

$and

ability to use an array of expressions. All expressions must test true.

// filtered: [ { name: 'Craig', state: 'MN' }]

[
  { name: "Craig", state: "MN" },
  { name: "Tim", state: "MN" },
  { name: "Joe", state: "CA" },
].filter(sift({ $and: [{ name: "Craig" }, { state: "MN" }] }));

$or

OR array of expressions.

// filtered: [ { name: 'Craig', state: 'MN' }, { name: 'Tim', state: 'MN' }]
[
  { name: "Craig", state: "MN" },
  { name: "Tim", state: "MN" },
  { name: "Joe", state: "CA" },
].filter(sift({ $or: [{ name: "Craig" }, { state: "MN" }] }));

$nor

opposite of or:

// filtered: [{ name: 'Joe', state: 'CA' }]
[
  { name: "Craig", state: "MN" },
  { name: "Tim", state: "MN" },
  { name: "Joe", state: "CA" },
].filter(sift({ $nor: [{ name: "Craig" }, { state: "MN" }] }));

$size

Matches an array - must match given size:

// filtered: ['food','cooking']
[{ tags: ["food", "cooking"] }, { tags: ["traveling"] }].filter(
  sift({ tags: { $size: 2 } }),
);

$type

Matches a values based on the type

[new Date(), 4342, "hello world"].filter(sift({ $type: Date })); // returns single date
[new Date(), 4342, "hello world"].filter(sift({ $type: String })); // returns ['hello world']

$regex

Matches values based on the given regular expression

["frank", "fred", "sam", "frost"].filter(
  sift({ $regex: /^f/i, $nin: ["frank"] }),
); // ["fred", "frost"]
["frank", "fred", "sam", "frost"].filter(
  sift({ $regex: "^f", $options: "i", $nin: ["frank"] }),
); // ["fred", "frost"]

$where

Matches based on some javascript comparison

[{ name: "frank" }, { name: "joe" }].filter(
  sift({ $where: "this.name === 'frank'" }),
); // ["frank"]
[{ name: "frank" }, { name: "joe" }].filter(
  sift({
    $where: function () {
      return this.name === "frank";
    },
  }),
); // ["frank"]

$elemMatch

Matches elements of array

var bills = [
  {
    month: "july",
    casts: [
      {
        id: 1,
        value: 200,
      },
      {
        id: 2,
        value: 1000,
      },
    ],
  },
  {
    month: "august",
    casts: [
      {
        id: 3,
        value: 1000,
      },
      {
        id: 4,
        value: 4000,
      },
    ],
  },
];

var result = bills.filter(
  sift({
    casts: {
      $elemMatch: {
        value: { $gt: 1000 },
      },
    },
  }),
); // {month:'august', casts:[{id:3, value: 1000},{id: 4, value: 4000}]}

$not

Not expression:

["craig", "tim", "jake"].filter(sift({ $not: { $in: ["craig", "tim"] } })); // ['jake']
["craig", "tim", "jake"].filter(sift({ $not: { $size: 5 } })); // ['tim','jake']

Date comparison

Mongodb allows you to do date comparisons like so:

db.collection.find({ createdAt: { $gte: "2018-03-22T06:00:00Z" } });

In Sift, you'll need to specify a Date object:

collection.find(
  sift({ createdAt: { $gte: new Date("2018-03-22T06:00:00Z") } }),
);

Custom behavior

Sift works like MongoDB out of the box, but you're also able to modify the behavior to suite your needs.

Custom operations

You can register your own custom operations. Here's an example:

import sift, { createEqualsOperation } from "sift";

var filter = sift(
  {
    $customMod: 2,
  },
  {
    operations: {
      $customMod(params, ownerQuery, options) {
        return createEqualsOperation(
          (value) => value % params !== 0,
          ownerQuery,
          options,
        );
      },
    },
  },
);

[1, 2, 3, 4, 5].filter(filter); // [1, 3, 5]

Omitting built-in operations

You can create a filter function that omits the built-in operations like so:

import { createQueryTester, $in, $all, $nin, $lt } from "sift";
const test = createQueryTester(
  {
    $eq: 10,
  },
  { operations: { $in, $all, $nin, $lt } },
);

[1, 2, 3, 4, 10].filter(test);

For bundlers like Webpack and Rollup, operations that aren't used are omitted from application bundles via tree-shaking.