jora

What is jora?

Jora is a powerful query language for JavaScript objects, allowing you to perform complex queries and transformations on JSON-like data structures. It is particularly useful for data manipulation, filtering, and aggregation.

What are jora's main functionalities?

Basic Querying

This feature allows you to extract specific fields from an array of objects. In this example, it extracts the 'name' field from each object in the array.

const jora = require('jora');
const data = [{ name: 'Alice', age: 30 }, { name: 'Bob', age: 25 }];
const result = jora('name')(data);
console.log(result); // ['Alice', 'Bob']

Filtering

This feature allows you to filter objects based on certain conditions. In this example, it filters out objects where the 'age' field is greater than 25.

const jora = require('jora');
const data = [{ name: 'Alice', age: 30 }, { name: 'Bob', age: 25 }];
const result = jora('[age > 25]')(data);
console.log(result); // [{ name: 'Alice', age: 30 }]

Aggregation

This feature allows you to perform aggregation operations on fields. In this example, it calculates the sum of the 'age' field across all objects.

const jora = require('jora');
const data = [{ name: 'Alice', age: 30 }, { name: 'Bob', age: 25 }];
const result = jora('age.sum()')(data);
console.log(result); // 55

Nested Queries

This feature allows you to perform queries on nested structures. In this example, it extracts the 'name' field from each object within the 'users' array.

const jora = require('jora');
const data = { users: [{ name: 'Alice', age: 30 }, { name: 'Bob', age: 25 }] };
const result = jora('users[name]')(data);
console.log(result); // ['Alice', 'Bob']

Other packages similar to jora

lodash

Lodash is a popular utility library that provides a wide range of functions for manipulating arrays, objects, and other data structures. While it does not offer a query language like Jora, it provides similar functionalities through its collection of utility functions.

json-query

Json-query is a lightweight library for querying JSON data. It offers a simpler syntax compared to Jora but is less powerful in terms of the complexity of queries it can handle.

jq

Jq is a command-line tool for processing JSON data. It offers a powerful query language similar to Jora but is designed to be used in a command-line environment rather than within JavaScript code.

README

Jora

JavaScript object query engine

STATUS: Jora is still very much work in progress. Syntax may change in next releases.

Features:

• Tolerant to data stucture queries (e.g. just returns nothing for paths that not reachable)
• Compact syntax for common tasks
• Aggregate values across arrays and eliminate duplicates by default
• Stat collecting mode (powers suggestions)
• Tolerant parsing mode (useful to provide suggestions for query in an editor)
• Extensible DSL on query build by custom method list

Related projects:

• jora-cli – Command line interface for Jora
• Jora sandbox – A Web interface where you can play with jora syntax or transform some JSON with zero setup
• Discovery – Uses jora to query a data for views
• JsonDiscovery – a browser’s extension for viewing JSON based on Discovery, available for Chrome and Firefox (read more in the article Changing a way we’re viewing JSON in a browser)

TODO:

• AST
• Immutable paths hoisting (reduce computations -> performance)
• Smart computation caching across queries
• Query parts performance stat
• Query transforming, e.g. query merge, subquery to a query, context inlining
• Method namespaces, e.g. semver, path, math etc
• Syntax highlighting
• Prettifier
• Move jison to dev dependencies
• Debugging (step by step evaluation)
• Input data shape prediction suitable for a query (based on touching paths)

More ideas and thoughts: Jora todo gist, Jora ToDo on Discovery.js projects state overview

Table of content:

• Install
• Quick demo
• API
  • Query introspection
• Syntax
  • Comments
  • Primitives
  • Keywords
  • Operators
  • Comparisons
  • Boolean logic
  • Block & definitions
  • Special references
  • Path chaining
  • Build-in methods
• License

Install

Install with npm:

npm install jora

Basic usage:

// ESM
import jora from 'jora';

// CommonJS
const jora = require('jora');

Bundles are available for use in a browser:

• dist/jora.js – minified IIFE with jora as global

<script src="node_modules/jora/dist/jora.js"></script>
<script>
  jora('query')(data, context);
</script>

• dist/jora.esm.js – minified ES module

<script type="module">
  import jora from 'node_modules/jora/dist/jora.esm.js'
  jora('query')(data, context);
</script>

One of CDN services like unpkg or jsDelivr can be used. By default (for short path) a ESM version is exposing. For IIFE version a full path to a bundle should be specified:

<!-- ESM -->
<script type="module">
  import jora from 'https://cdn.jsdelivr.net/npm/jora';
  import jora from 'https://unpkg.com/jora';
</script>

<!-- IIFE with an export `jora` to global -->
<script src="https://cdn.jsdelivr.net/npm/jora/dist/jora.js"></script>
<script src="https://unpkg.com/jora/dist/jora.js"></script>

Quick demo

Get npm dependency paths (as a tree) that have packages with more than one version:

import jora from 'jora';
import { exec } from 'child_process';

function printTree() {
    // see implementation in examples/npm-ls.js
}

exec('npm ls --all --json', (error, stdout) => {
    if (error) {
        return;
    }

    const npmTree = JSON.parse(stdout);
    const depsPathsToMultipleVersionPackages = jora(`
        $normalizedDeps: => dependencies.entries().({ name: key, ...value });
        $multiVersionPackages:
            ..$normalizedDeps()
            .group(=>name, =>version)
            .({ name: key, versions: value.sort() })
            .[versions.size() > 1];

        $pathToMultiVersionPackages: => .($name; {
            name,
            version,
            otherVersions: $multiVersionPackages[=>name=$name].versions - version,
            dependencies: $normalizedDeps()
                .$pathToMultiVersionPackages()
                .[name in $multiVersionPackages.name or dependencies]
        });

        $pathToMultiVersionPackages()
    `)(npmTree);

    printTree(depsPathsToMultipleVersionPackages);
});

Example of output:

jora@1.0.0
├─ c8@7.11.0
│  ├─ istanbul-lib-report@3.0.0
│  │  └─ supports-color@7.2.0 [more versions: 8.1.1]
│  ├─ test-exclude@6.0.0
│  │  └─ minimatch@3.1.2 [more versions: 3.0.4]
│  ├─ v8-to-istanbul@8.1.1
│  │  └─ convert-source-map@1.8.0
│  │     └─ safe-buffer@5.1.2 [more versions: 5.2.1]
│  ├─ yargs-parser@20.2.9 [more versions: 20.2.4]
│  └─ yargs@16.2.0
│     └─ yargs-parser@20.2.9 [more versions: 20.2.4]
├─ eslint@8.10.0
│  ├─ @eslint/eslintrc@1.2.0
│  │  ├─ ignore@4.0.6 [more versions: 5.2.0]
│  │  └─ minimatch@3.1.2 [more versions: 3.0.4]
...

API

import jora from 'jora';

// create a query
const query = jora('foo.bar');
// or with custom methods
const queryWithCustomMethods = jora.setup({
    myMethod(current) { /* do something and return a new value */ }
});

// perform a query
const result = query(data, context);
const result = queryWithCustomMethods('foo.myMethod()')(data, context);

Options:

• methods

  Type: Object
  Default: undefined

  Additional methods for using in query passed as an object, where a key is a method name and a value is a function to perform an action. It can override build-in methods.

• debug

  Type: Boolean or function(name, value)
  Default: false

  Enables debug output. When set a function, this function will recieve a section name and its value.

• tolerant

  Type: Boolean
  Default: false

  Enables tolerant parsing mode. This mode supresses parsing errors when possible.

• stat

  Type: Boolean
  Default: false

  Enables stat mode. When mode is enabled a query stat interface is returning instead of resulting data.

Query introspection

To introspect a query, it should be compiled in "stat" (statistic) mode by passing a stat option. In this case a result of the query evaluation will be a special API with encapsulated state instead of a value:

import jora from 'jora';

const query = jora('...query...', { stat: true });
const statApi = query(data);
// { stat() { ... }, suggestion() { ... }, ... }

The returned API allows fetching the values which are passed through a location in a query (the stat() method) as well as a list of suggestions for a location (the suggestion() method):

import jora from 'jora';

const query = jora('.[foo=""]', { stat: true });
const statApi = query([{ id: 1, foo: "hello" }, { id: 2, foo: "world" }]);

statApi.stat(3);
// [
//   {
//     context: 'path',
//     from: 2,
//     to: 5,
//     text: 'foo',
//     values: Set(2) { [Object], [Object] },
//     related: null
//   }
// ]

statApi.suggestion(3); // .[f|oo=""]
// [
//   {
//     type: 'property',
//     from: 2,
//     to: 5,
//     text: 'foo',
//     suggestions: [ 'id', 'foo' ]
//   }
// ]

statApi.suggestion(7); // .[foo="|"]
// [
//   {
//     type: 'value',
//     from: 6,
//     to: 8,
//     text: '""',
//     suggestions: [ 'hello', 'world' ]
//   }
// ]

That's an effective way to use stat mode together with tolerant mode for incomplete queries:

import jora from 'jora';

const query = jora('.[foo=]', {
    stat: true,
    tolerant: true // without the tolerant option a query compilation
                   // will raise a parse error:
                   // .[foo=]
                   // ------^
});
const statApi = query([{ id: 1, foo: "hello" }, { id: 2, foo: "world" }]);

statApi.suggestion(6); // .[foo=|]
// [
//   {
//     type: 'value',
//     from: 6,
//     to: 6,
//     text: '',
//     suggestions: [ 'hello', 'world' ]
//   },
//   {
//     type: 'property',
//     from: 6,
//     to: 6,
//     text: '',
//     suggestions: [ 'id', 'foo' ]
//   }
// ]

Methods

• stat(pos: number, includeEmpty?: boolean)

  Returns an array of ranges with all the values which are passed through pos during performing a query.

  Output format:

      suggestion(): Array<{
          context: 'path' | 'key' | 'value' | 'in-value' | 'value-subset' | 'var',
          from: number,
          to: number,
          text: string,
          values: Set<any>,
          related: Set<any> | null
      }> | null
  

• suggestion(pos: number, options?)

  Returns suggesion values grouped by a type or null if there is no any suggestions. The following options are supported (all are optional):

  • limit (default: Infinity) – a max number of the values that should be returned for each value type ("property", "value" or "variable")
  • sort (default: false) – a comparator function (should take 2 arguments and return a negative number, 0 or a positive number) for value list sorting, makes sence when limit is used
  • filter (default: function) – a filter function factory (pattern => value => <expr>) to discard values from the result when returns a falsy value (default is equivalent to patttern => value => String(value).toLowerCase().includes(pattern))

  Output format:

      suggestion(): Array<{
          type: 'property' | 'value' | 'variable',
          from: number,
          to: number,
          text: string,
          suggestions: Array<string | number>
      }> | null
  

Syntax

Comments

// single-line comment
/* multi-line
comment */

Primitives

Jora	Description
42 -123 4.22 1e3 1e-2	Numbers
0xdecaf -0xC0FFEE	Hexadecimal numbers
"string" 'string'	Strings
`template line1 template line2` `template ${hello} ${world}`	Template
/regexp/ /regexp/i	A JavaScript regexp, only i flag supported
{ }	Object initializer/literal syntax. Spread operator (...) can be used, e.g. { a: 1, ..., ...foo } (... with no expression on right side the same as ...$)
[ ]	Array initializer/literal syntax. Spread operator (...) can be used, e.g. [1, ..., ...foo] (... with no expression on right side the same as ...$). Unlike JavaScript, spread operator in jora inlines arrays only and left as is any other values, i.e. [...[1, 2], ...3, ..."45", { "6": 7 }] -> [1, 2, 3, "45", { "6": 7 }]
=> e < block > (deprecated)	A function NOTE: Syntax < block > is deprecated, avoid to use it
query asc query desc query asc, query desc, ...	A sorting function that takes two arguments and compare query result for each in specified order (asc – ascending, desc – descending)
query ascN query descN	The same as asc/desc but natural sorting
query ascA query descA	The same as asc/desc but reverse order for numbers
query ascAN query descAN	The same as asc/desc but natural sorting and reverse order for numbers

Keywords

Following keywords can be used with the same meaning as in JavaScript:

• true
• false
• null
• undefined
• Infinity
• NaN

Operators

Jora	Description
x + y	Add In case one of the operands is an array it produces new array with elements from `x` and `y` excluding duplicates
x - y	Subtract In case one of the operands is an array with elements from `x` excluding elements from `y`
x * y	Multiply
x / y	Divide
x % y	Modulo

Comparisons

Jora	Description
x = y	Equals (as === in JS)
x != y	Not equals (as !== in JS)
x < y	Less than
x <= y	Less than or equal to
x > y	Greater than
x >= y	Greater than or equal to
x ~= y	Match operator, behaviour depends on y type: RegExp – test against regexp function – test like filter() null or undefined – always truthy anything else – always falsy

Boolean logic

Jora	Description
( x )	Explicity operator precedence. Definitions are allowed (i.e. ($a: 1; $a + $a) see bellow)
x or y	Boolean or. Equivalent to || in JS, but x tests with bool() method
x and y	Boolean and. Equivalent to && in JS, but x tests with bool() method
not x no x	Boolean not. Equivalent to && in JS, but x tests with bool() method
x ? y : z	If x is truthy than return y else return z. x tests with bool() method
x in [a, b, c] [a, b, c] has x	Equivalent to x = a or x = b or x = c
x not in [a, b, c] [a, b, c] has no x	Equivalent to x != a and x != b and x != c

Block & definitions

Some constructions suppose to use a block, which may consists of a variable definition list (should comes first) and an expression. Both are optional. When an expression is empty, a current value (i.e. $) returns.

The syntax of definition (white spaces between any part are optional):

$ident ;
$ident : expression ;

For example:

$foo:123;          // Define `$foo` variable
$bar;              // The same as `$bar:$.bar;` or `$a: bar;`
$baz: $foo + $bar; // Definitions may be used in following expressions

In terms of JavaScript, a block creates a new scope. Once a variable is defined, its value never change. Variables can be redefined in nested scopes, but can't be duplicated in the same scope - it causes to error.

Special references

Jora	Description
$	A scope input data (current value). On top level scope it's the same as @. In most cases it may be omitted. Used implicitly an input for subquery when no other subjects is defined (e.g. foo() and .foo() are equivalent for $.foo()).
$$	A reference to the second parameter of closest function or undefined when no such
@	A query input data
#	A query context

Since Jora's query performs as query(data, context), in terms of Jora it looks like query(@, #).

Path chaining

jora	Description
ident	The same as $.ident
.ident	Child member operator (example: foo.bar.baz, #.foo['use any symbols for name'])
..ident ..( block )	Recursive descendant operator (example: ..deps, ..(deps + dependants))
.[ block ]	Filter a current data. Equivalent to a .filter(<block>)
.( block )	Map a current data. Equivalent to a .map(<block>)
method() .method() ..$method()	Invoke a method to current value, where $method is a reference to definition value (i.e. $example: => $ * 10; 2.$plural(["example", "examples"])). Can take arguments (i.e. $method(one, 2)).
$method() .$method() ..method()	Invoke a method to current value. See build-in methods below
path[expr]	Array-like notation to access properties. Behaves like pick() method. In case you need to fetch a value to each element of array use .($[expr]) or map(=>$[expr])
[from:to] [from:to:step]	Slice notation. Examples: $str: '<foo>'; str[1:-1] (result is 'foo') or $ar:[1,2,3,4,5,6]; $ar[-3::-1] (result is [6,5,4])
expr | [definitions] expr | ...	Pipeline operator. It's useful to make a query value as current value. Approximately this effect can be obtained using variables: $ar: [1,2,3]; { size: $ar.size(), top2: $ar[0:2] }. However, with pipeline operator it's a bit simplier and clear: `[1,2,3]

Build-in methods

jora	Description
bool()	The same as Boolean() in JS, with exception that empty arrays and objects with no keys treats as falsy
keys()	The same as Object.keys() in JS
values()	The same as Object.values() in JS
entries()	Similar to Object.entries() in JS with a difference: { key, value } objects is using for entries instead of array tuples
fromEntries()	Similar to Object.fromEntries() in JS with difference: { key, value } objects are expecting as entries instead of array tuples
pick("key") pick(index) pick(fn)	Get a value by a key, an index or a function. It returns an element with e index for arrays, a char with e index for strings, and a value with e key (must be own key) for enything else. Negative indecies are supported for arrays and strings. Current value is element for an array, a char for a string or an entry value for object. Arg1 (i.e. $$) is an index for arrays and strings, and a key for objects.
size()	Returns count of keys if current data is object, otherwise returns length value or 0 when field is absent
sort(<fn>)	Sort an array by a value fetched with getter (<fn>). Keep in mind, you can use sorting function definition syntax using asc and desc keywords, qhich is more effective in many ways. In case of sorting function definition usage, < and > are not needed and you can specify sorting order for each component. Following queries are equivalents: sort(<foo.bar>) and sort(foo.bar asc) sort(<foo>).reverse() and sort(foo desc) sort(<[a, b]>) and sort(a asc, b asc)
reverse()	Reverse order of items
group(<fn>[, <fn>])	Group an array items by a value fetched with first getter.
filter(<fn>)	The same as Array#filter() in JS
map(<fn>)	The same as Array#map() in JS
split(pattern)	The same as String#split() in JS. pattern may be a string or regexp
join(separator)	The same as Array#join() in JS. When separator is undefined then "," is using
slice(from, to)	The same as Array#slice() or String#slice() in JS
match(pattern, matchAll?)	Similar to String#match(). pattern might be a RegExp or string. When matchAll is truthy then returns an array of all occurrences of the pattern. Expressions match(/../g) and match(/../, true) are equivalent.
reduce(fn[, initValue])	The same as Array#reduce() in JS. Use $$ to access to accumulator and $ to current value, e.g. find the max value reduce(=>$ > $$ ? $ : $$)

License

MIT

Changelog

1.0.0-beta.6 (April 18, 2022)

• Added support for custom methods as a string (jora query) in jora.setup()
• Added a check for all methods are defined on query compilation step to avoid runtime exceptions
• Allowed g, m, s and u flags in regexp literals
• Improved error message when a methods is not defined, i.e. Method "foo" is not defined instead of m.foo is not a function
• Improved error locations in string literals
• Disallowed a backslash before closing quote or apostrophe in string literals
• Fixed match() method to work well for RegExp with g flag and for strings when matchAll is true
• Fixed sort() method to perform a stable sort for old js-engines and always place undefined values last
• Fixed range in details for bad input errors
• Fixed suggestion support in template literals (#33)
• Fixed suggestions for = and != operators by avoiding unfold array values
• Fixed suggestions for arrays in in, not in, has and has no operators to exclude literal values only
• Removed value suggestions in cases | in <string or number> and <string or number> has |
• Fixed a call stack overflow exception when too many (over ~110k) suggestion values for a range
• BREAKING Reworked stat API:
  • Changed stat() API method to return values as is, i.e. a Set instance instead of its materialization as an array. Mutations of sets (values and related) should be avoided since they are shared between all stat API methods calls.
  • Changed a signature of suggestion() API method to get options as a second argument (optional), with the following options:
    • limit (default: Infinity) – a max number of the values that should be returned for each value type ("property", "value" or "variable")
    • sort (default: false) – a comparator function (should take 2 arguments and return a negative number, 0 or a positive number) for value list sorting, makes sence when limit is used
    • filter (default: function) – a filter function factory (pattern => value => <expr>) to discard values from the result when returns a falsy value (default is equivalent to patttern => value => String(value).toLowerCase().includes(pattern))
  • Changed suggestion() API method result to return values grouped by a type:

        suggestion(): Array<{
            type: 'property' | 'value' | 'variable',
            from: number,
            to: number,
            text: string,
            suggestions: Array<string | number>
        }> | null
    

  • All the changes are targeted to improve performance and memory consumption when a query is performing to a huge datasets (hundreds of thousands of values). As a result a suggestions fetching is boosted up to 30-40 times for such cases.
• Converted to Dual Package, i.e. ESM and CommonJS support
• Changed Node.js support to ^10.12.0 || ^12.20.0 || ^14.13.0 || >=15.0.0
• Changed dist modules:
  • Removed jora.min.js
  • Changed jora.js format from CJS to IIFE and applied minification
  • Added jora.esm.js (minified ESM module)
  • Added source maps jora.js.map & jora.esm.js.map