mingo

What is mingo?

Mingo is a JavaScript library that provides MongoDB-like query and aggregation capabilities for in-memory data processing. It allows you to filter, sort, group, and transform data collections using a syntax similar to MongoDB queries.

What are mingo's main functionalities?

Querying

This feature allows you to perform complex queries on your data collections. The example demonstrates how to filter a list of people to find those older than 26.

const mingo = require('mingo');
const data = [
  { name: 'John', age: 25 },
  { name: 'Jane', age: 30 },
  { name: 'Jim', age: 27 }
];
const query = new mingo.Query({ age: { $gt: 26 } });
const result = query.find(data).all();
console.log(result); // Output: [ { name: 'Jane', age: 30 }, { name: 'Jim', age: 27 } ]

Aggregation

This feature allows you to perform aggregation operations on your data collections. The example demonstrates how to calculate the average salary of people older than 26.

const mingo = require('mingo');
const data = [
  { name: 'John', age: 25, salary: 50000 },
  { name: 'Jane', age: 30, salary: 60000 },
  { name: 'Jim', age: 27, salary: 55000 }
];
const pipeline = [
  { $match: { age: { $gt: 26 } } },
  { $group: { _id: null, averageSalary: { $avg: '$salary' } } }
];
const result = mingo.aggregate(data, pipeline);
console.log(result); // Output: [ { _id: null, averageSalary: 57500 } ]

Projection

This feature allows you to project specific fields from your data collections. The example demonstrates how to retrieve only the 'name' and 'age' fields from a list of people.

const mingo = require('mingo');
const data = [
  { name: 'John', age: 25, salary: 50000 },
  { name: 'Jane', age: 30, salary: 60000 },
  { name: 'Jim', age: 27, salary: 55000 }
];
const query = new mingo.Query({});
const result = query.find(data).project({ name: 1, age: 1 }).all();
console.log(result); // Output: [ { name: 'John', age: 25 }, { name: 'Jane', age: 30 }, { name: 'Jim', age: 27 } ]

Other packages similar to mingo

lodash

Lodash is a JavaScript utility library that provides a wide range of functions for manipulating arrays, objects, and other data structures. While it does not offer MongoDB-like query syntax, it provides similar functionalities for filtering, sorting, and transforming data collections.

underscore

Underscore is another JavaScript utility library similar to Lodash. It provides a set of utility functions for working with arrays, objects, and other data types. Like Lodash, it does not offer MongoDB-like query syntax but provides similar data manipulation capabilities.

json-query

json-query is a library for querying JSON data structures using a simple query language. It provides functionalities for filtering and transforming JSON data, similar to Mingo, but with a different query syntax.

README

mingo

MongoDB query language for in-memory objects

Install

$ npm install mingo

Features

• Supports Dot Notation for both <array>.<index> and <document>.<field> selectors
• Query and Projection Operators
  • Array Operators
  • Bitwise Operators
  • Comparisons Operators
  • Element Operators
  • Evaluation Operators
  • Logical Operators
  • Projection Operators
• Aggregation Framework Operators
  • Pipeline Operators
  • Accumulator Operators
  • Expression Operators
    • Arithmetic Operators
    • Array Operators
    • Boolean Operators
    • Comparisons Operators
    • Conditional Operators
    • Date Operators
    • Literal Operators
    • Miscellaneous Operators
    • Object Operators
    • Set Operators
    • String Operators
    • Trignometry Operators
    • Type Operators
    • Variable Operators
• Support for adding custom operators using mingo.addOperators
• Match against user-defined types
• Support for aggregaion variables
  • $$ROOT,$$CURRENT,$$DESCEND,$$PRUNE,$$KEEP,$$REMOVE
• ES6 module compatible
• Support integrating with custom collections via mixin
• Query filtering and aggregation streaming.

For documentation on using query operators see mongodb

Documentation

• Package docs
• Custom Operators

Usage

// Use as es6 module
import mingo from 'mingo'

// or vanilla nodeJS
const mingo = require('mingo')

Change in 4.x.x

The $where operator is not loaded by default and must be explicitly registered if required. See documentation for preferred alternatives.

Changes in 3.0.0

Default exports and operators

The default export of the main module only includes Aggregator, Query, aggregate(), find(), and remove().

Only Query and Projection operators are loaded by default when using the main module. This is done using the side-effect module mingo/init/basic, and also automatically includes pipeline operators $project, $skip, $limit, and $sort.

If your application uses most of the available operators or you do not care about bundle size, you can load all operators as shown below.

// Note that doing this effectively imports the entire library into your bundle and unused operators cannot be tree shaked
import 'mingo/init/system'

Or from the node CLI

node -r 'mingo/init/system' myscript.js

Custom Operators

The addOperators function for registering custom operators and helper constants have been moved to mingo/core. The constants OP_XXX have been deprecated and replace with an enum type OperatorType also in mingo/core. The values defined include;

• ACCUMULATOR
• EXPRESSION
• PIPELINE
• PROJECTION
• QUERY

Lastly, the function argument to addOperators(operatorType, fn) now accepts an object with the these two internal functions;

• computeValue(obj: AnyVal, expr: AnyVal, operator: string, options?: ComputeOptions): AnyVal
• resolve(obj: AnyVal, selector: string, options?: ResolveOptions): AnyVal

Any extra utility may be imported directly from the specific module.

Importing submodules

Submodule imports are supported for both ES6 and ES5.

The following two examples are equivalent.

ES6

This work natively in typescript since it knows how to load commonJS modules as ES6. You may optionally install the esm module to use this syntax.

import { $unwind } from 'mingo/operators/pipeline'

ES5

Unlike the ES6 version, it is necessary to specify the operator module in the path to avoid loading any extras

const $unwind = require('mingo/operators/pipeline/unwind').$unwind

Configuration

To support tree-shaking, you may import and register specific operators that will be used in your application.

import { useOperators, OperatorType } from 'mingo/core'
import { $trunc } from 'mingo/operators/expression'
import { $bucket } from 'mingo/operators/pipeline'

useOperators(OperatorType.EXPRESSION, { $trunc, $floor })
useOperators(OperatorType.PIPELINE, { $bucket })

Using query object to test objects

import { Query } from 'mingo'

// create a query with criteria
// find all grades for homework with score >= 50
let query = new Query({
  type: "homework",
  score: { $gte: 50 }
});

// test if an object matches query
query.test(doc)

Searching and Filtering

import { Query } from 'mingo'

// input is either an Array or any iterable source (i.e Object{next:Function}) including ES6 generators.
let criteria = { score: { $gt: 10 } }

let query = new Query(criteria)

// filter collection with find()
let cursor = query.find(collection)

// alternatively use shorthand
// cursor = mingo.find(collection, criteria)

// sort, skip and limit by chaining
cursor.sort({student_id: 1, score: -1})
  .skip(100)
  .limit(100)

// count matches. exhausts cursor
cursor.count()

// classic cursor iterator (old school)
while (cursor.hasNext()) {
  console.log(cursor.next())
}

// ES6 iterators (new cool)
for (let value of cursor) {
  console.log(value)
}

// all() to retrieve matched objects. exhausts cursor
cursor.all()

Using $jsonSchema operator

To use the $jsonSchema operator, you must register your own JsonSchemaValidator in the options. No default implementation is provided out of the box so users can use a library with their preferred schema format.

The example below uses Ajv to implement schema validation.

import { RawObject } from "mingo/types"
import { JsonSchemaValidator } from "mingo/core"
import Ajv, { Schema } from "ajv"

const jsonSchemaValidator: JsonSchemaValidator = (s: RawObject) => {
  const ajv = new Ajv();
  const v = ajv.compile(s as Schema);
  return (o: RawObject) => (v(o) ? true : false);
};

const schema = {
  type: "object",
  required: ["item", "qty", "instock"],
  properties: {
    item: { type: "string" },
    qty: { type: "integer" },
    size: {
      type: "object",
      required: ["uom"],
      properties: {
        uom: { type: "string" },
        h: { type: "number" },
        w: { type: "number" },
      },
    },
    instock: { type: "boolean" },
  },
};

// queries documents using schema validation
find(docs, { $jsonSchema: schema }, {}, { jsonSchemaValidator }).all();

Note: An error is thrown when the $jsonSchema operator is used without a the jsonSchemaValidator configured.

Aggregation Pipeline

import { Aggregator } from 'mingo/aggregator'
import { useOperators, OperatorType } from 'mingo/core'
import { $match, $group } from 'mingo/operators/pipeline'
import { $min } from 'mingo/operators/accumulator'

// ensure the required operators are preloaded prior to using them.
useOperators(OperatorType.PIPELINE, { $match, $group })
useOperators(OperatorType.ACCUMULATOR, { $min })

let agg = new Aggregator([
  {'$match': { "type": "homework"}},
  {'$group': {'_id':'$student_id', 'score':{$min:'$score'}}},
  {'$sort': {'_id': 1, 'score': 1}}
])

// return an iterator for streaming results
let stream = agg.stream(collection)

// return all results. same as `stream.all()`
let result = agg.run(collection)

Benefits

• Better alternative to writing custom code for transforming collection of objects
• Quick validation of MongoDB queries without the need for a database
• MongoDB query language is among the best in the market and is well documented

Contributing

• Squash changes into one commit
• Run npm test to build and execute unit tests
• Submit pull request

License

MIT

Changelog

4.4.1 / 2021-11-16

Fixes

• Remove lodash reference in expression operators.