rudder-json-template-engine - npm Package Compare versions

Comparing version 0.5.2 to 0.5.3

build/types.d.ts

@@ -10,3 +10,4 @@ export declare enum Keyword {
     ASYNC = "async",
-    IN = "in"
+    IN = "in",
+    NOT = "not"
 }
@@ -13,0 +14,0 @@ export declare enum TokenType {

build/types.js

@@ -15,2 +15,3 @@ "use strict";
     Keyword["IN"] = "in";
+    Keyword["NOT"] = "not";
 })(Keyword = exports.Keyword || (exports.Keyword = {}));
@@ -17,0 +18,0 @@ var TokenType;

CHANGELOG.md

 # Changelog
 
+## [0.5.3](https://github.com/rudderlabs/rudder-json-template-engine/compare/v0.5.2...v0.5.3) (2023-09-07)
+
+
+### Miscellaneous
+
+* update readme docs ([140f6a2](https://github.com/rudderlabs/rudder-json-template-engine/commit/140f6a2d7566ac77e0c918e70289b899ab54de24))
+
 ## [0.5.2](https://github.com/rudderlabs/rudder-json-template-engine/compare/v0.5.1...v0.5.2) (2023-06-26)
@@ -4,0 +11,0 @@

package.json

 {
   "name": "rudder-json-template-engine",
-  "version": "0.5.2",
+  "version": "0.5.3",
   "homepage": "https://github.com/rudderlabs/rudder-json-template-engine",
@@ -5,0 +5,0 @@ "description": "A library for evaluating JSON template expressions.",

readme.md

@@ -21,9 +21,24 @@ <p align="center">
 
-# rudder-json-template-engine
-## Motivation
-We are an integration platform and support 200+ integrations. We implemented these integrations using native Javascript code to transform incoming events to destination payload, so in summary, it is JSON data manipulation. Maintaining all these integrations is challenging, so we explored [jsonata](https://github.com/jsonata-js/jsonata) to write less code to transform JSON data. While this library is excellent, we still need to meet our performance needs. For example, JSONata parses the given template, creates an Abstract Syntax Tree (AST), and interprets the AST for the given input. Since we need to traverse the AST every time, it is slow, so we wanted to build a template engine that generates Javascript code so there will be less overhead during the runtime.
-
+# JSON Template Engine
 ## Overview
-A library to process JSON data using a custom syntax based on javascript and [jspath](https://github.com/dfilatov/jspath). We thank the jspath authors for their excellent work, as our library is an extension of the original library. We also want to thank [IBM](https://www.ibm.com/) team for their work on [jsonata](https://github.com/jsonata-js/jsonata), as we have taken several ideas from the library. You can also consider our library as an alternative to [jsonata](https://github.com/jsonata-js/jsonata).
 
+Welcome to our JSON Template Engine! This powerful tool simplifies transforming JSON data from one format to another, making it easier to manage and maintain complex integrations. 
+
+### Why JSON Template Engine?
+
+As an integration platform supporting over 200 integrations, we understand the challenges of maintaining and optimizing these connections. Traditionally, we used native JavaScript code for data transformation, which required significant effort and maintenance. While JSONata offered a more efficient way to manipulate JSON data, we still encountered performance bottlenecks due to its parsing and interpretation overhead.
+
+### Our Solution
+
+To address these challenges, we've developed our own JSON Transformation Engine. This engine generates optimized JavaScript code from transformation templates, reducing runtime overhead and significantly improving performance.
+
+## Key Features
+
+- **Efficiency**: Our engine generates JavaScript code that minimizes parsing and interpretation overhead, ensuring faster execution.
+
+- **Extensibility**: Easily add new transformation templates to meet your specific integration needs.
+
+- **Simplicity**: Write concise transformation templates that are easy to understand and maintain.
+  
+## Implementation
 This library generates a javascript function code from the template and then uses the function to evaluate the JSON data. It outputs the javascript code in the following stages:
@@ -46,21 +61,228 @@ 1. [Lexing](src/lexer.ts) (Tokenization)
 ## Features
-1. [Variables](test/scenarios/assignments/template.jt)
-1. [Arrays](test/scenarios//arrays/template.jt)
-1. [Objects](test/scenarios/objects/template.jt)
-1. [Functions](test/scenarios/functions/template.jt)
-1. [Bindings](test/scenarios/bindings/template.jt)
-1. [Comments](test/scenarios/comments/template.jt)
-1. [Paths](test/scenarios/paths/template.jt)
-    * [Filters](test/scenarios/filters)
-    * [Selectors](test/scenarios/selectors)
-    * [Context variables](test/scenarios/context_variables/template.jt)
-    * [Simple paths](test/scenarios/paths/simple_path.jt)
-    * [Rich paths](test/scenarios/paths/rich_path.jt)
-    * [Paths options](test/scenarios/paths/options.jt)
-1. [Conditions](test/scenarios/conditions/template.jt)
-    * [Comparisons](test/scenarios/comparisons/template.jt)
-1. [Math operations](test/scenarios/math/template.jt)
-1. [Logical operations](test/scenarios/logics/template.jt)
-1. [Compile time expressions](test/scenarios/compile_time_expressions/template.jt)
+Template is a set of statements and result the last statement is the output of the template.
 
+### Variables
+```js
+const a = 1
+let b = a + 2
+a + b
+```
+Refer this [example](test/scenarios/assignments/template.jt) for more details.
+
+### Basic Expressions
+#### Conditions
+```js
+a > b ? a : c
+```
+Refer this [example](test/scenarios/conditions/template.jt) for more details.
+
+#### Comparisons
+```js
+(a === b || c > d)
+```
+Refer this [example](test/scenarios/comparisons/template.jt) for more details.
+
+#### Math Operations
+```js
+10 - 2 + (2 * 10)
+```
+Refer this [example](test/scenarios/math/template.jt) for more details.
+
+#### Logical operations
+```js
+false || true 
+```
+Refer this [example](test/scenarios/logics/template.jt) for more details.
+### Input and Bindings
+Input refers to the JSON document we would like to process using a template. Bindings refer to additional data or functions we would provide to process the data efficiently.
+
+Example:
+* Template: `"Hello " + (.name ?? $.defaultName)`
+* Evaluation: `engine.evaluate({name: 'World'}, {defaultName: 'World'});`
+* `{name: 'World'}` is input.
+  * `^.name` refers to "name" property of the input. We can also use `.name` to refer the same. `^` always refers to the root of the input and `.` refers to current context. Refer this [example](test/scenarios/selectors/context_variables.jt) for more details.
+* `{defaultName: 'World'}` is bindings.
+  * `$.defaultName` refers to "defaultName" property of the bindings. Refer this [example](test/scenarios/bindings/template.jt) for more details.
+
+### Arrays
+```js
+let arr = [1, 2, 3, 4]
+let a = arr[1, 2] // [2, 3]
+let b = arr[0:2] // [1, 2]
+let c = arr[-2:] // [3, 4]
+```
+Refer this [example](test/scenarios/arrays/template.jt) for more details.
+
+### Objects
+```js
+let key = "some key"
+// { "a": 1, "b": 2, "c": 3, "some key": 4 }
+let obj = {a: 1, b: 2, c: 3, [key]: 4 } 
+let a = obj["a"] // 1
+let b = obj.a // 1
+let c = obj{["a", "b"]} // { "a": 1, "b": 2}
+let d = obj{~["a", "b"]} // { "c": 3, "some key": 4}
+```
+Refer this [example](test/scenarios/objects/template.jt) for more details.
+
+### Functions
+#### Normal functions
+```js
+let fn = function(arg1, arg2){
+  arg1 + arg2
+}
+```
+The result of the last statement of function will be returned as result of the function. We can also use rest params (`...args`).
+#### Lambda/Short functions
+```js
+let fn = array.map(lambda 2 * ?0);
+```
+This function gets converted to:
+```js
+let fn = array.map(function(args) {
+  2 * args[0]
+})
+```
+Lambda functions are short to express the intention and it is convenient sometimes.
+#### Async functions
+```js
+let fn = async function(arg1, arg2){
+  const result = await doSomethingAsync(arg1, arg2)
+  doSomethingSync(result)
+}
+```
+**Note:** When we want to use async functions then we need to create template engine using `JsonTemplateEngine.create`. If you create a template this way then it will be created as an async function so we can `await` anywhere in the template.
+```js
+let result = await doSomething(.a, .b)
+```
+Refer this [example](test/scenarios/functions/template.jt) for more details.
+
+### Paths
+Paths are used to access properties in `input`, `bindings` and `variables`.
+#### Simple Paths
+Simple paths support limited path features and get translated as direct property access statements in the generate javascript code.
+`a.b.c` gets translated to `a?.b?.c` so they are very fast compared to [Rich paths](#rich-paths). Simple paths are ideal when we know the object structure.
+
+**Supported features:**
+* [Simple Selectors](#simple-selectors)
+* [Single Index Filters](#single-index-or-property-filters)
+Refer this [example](test/scenarios/paths/simple_path.jt) for more details.
+#### Rich Paths
+Rich paths gets converted complex code to support different variations in the data.
+
+If we use this rich path`~r a.b.c` then it automatically handles following variations.
+* `[{"a": { "b": [{"c": 2}]}}]`
+* `{"a": { "b": [{"c": 2}]}}`
+* `{"a": [{ "b": [{"c": 2}]}]}`
+Refer this [example](test/scenarios/paths/rich_path.jt) for more details.
+#### Simple selectors
+```js
+let x = a.b.c;
+let y = a."some key".c
+```
+Refer this [example](test/scenarios/selectors/template.jt) for more details.
+#### Wildcard selectors
+```js
+a.*.c // selects c from any direct property of a
+```
+Refer this [example](test/scenarios/selectors/wild_cards.jt) for more details.
+#### Descendent selectors
+```js
+// selects c from any child property of a
+// a.b.c, a.b1.b2.c or a.b1.b2.b3.c
+let x = a..c; 
+let y = a.."some key";
+```
+Refer this [example](test/scenarios/selectors/template.jt) for more details.
+#### Single Index or Property Filters
+```js
+let x = a[0].c; 
+let y = a[-1].c;  // selects last element from array
+let z = a["some key"].c
+```
+Refer this [example](test/scenarios/filters/array_filters.jt) for more details.
+#### Multi Indexes or Properties Filters
+```js
+let x = a[0, 2, 5].c; 
+let y = a["some key1", "some key2"].c;
+```
+Refer this [example](test/scenarios/filters/array_filters.jt) for more details.
+#### Range filters
+```js
+let x = a[2:5].c; 
+let y = a[:-2].c; 
+let z = a[2:].c; 
+```
+#### Object Property Filters
+```js
+let x = obj{["a", "b"]};  // selects a and b
+let y = obj{~["a", "b"]}; // selects all properties except a and b
+```
+Refer this [example](test/scenarios/filters/object_indexes.jt) for more details.
+#### Conditional or Object Filters
+```js
+let x = obj{.a > 1};
+```
+Refer this [example](test/scenarios/filters/object_filters.jt) for more details.
+#### Block expressions
+```js
+let x = obj.({
+  a: .a + 1,
+  b: .b + 2
+});
+let x = obj.([.a+1, .b+2]);
+```
+Refer this [example](test/scenarios/paths/block.jt) for more details.
+
+#### Context Variables
+```js
+.orders@order#idx.products.({
+    name: .name,
+    price: .price,
+    orderNum: idx,
+    orderId: order.id
+})
+```
+Use context variables: `@order` and `#idx`, we can combine properties of orders and products together. Refer this [example](test/scenarios/context_variables/template.jt) for more details.
+#### Path Options
+We can mention defaultPathType while creating engine instance.
+```js
+// For using simple path as default path type
+// a.b.c will be treated as simple path
+JsonTemplateEngine.create(`a.b.c`, {defaultPathType: PathType.SIMPLE});
+// For using rich path as default path type
+// a.b.c will be treated as rich path
+JsonTemplateEngine.create(`a.b.c`, {defaultPathType: PathType.RICH});
+```
+We can override the default path option using tags.
+```js
+// Use ~s to treat a.b.c as simple path
+~s a.b.c 
+// Use ~r to treat a.b.c as rich path
+~r a.b.c
+```
+**Note:** Rich paths are slower compare to the simple paths.
+Refer this [example](test/scenarios/paths/options.jt) for more details.
+
+### Compile time expressions
+Compile time expressions are evaluated during compilation phase using compileTimeBindings option.
+```js
+// {{$.a.b.c}} gets translated to 1 and
+// final translated code will be "let a = 1;"
+JsonTemplateEngine.create(`let a = {{$.a.b.c}};`, {
+  compileTimeBindings: {
+    a: {
+      b: {
+        c: 1
+      }
+    }
+  }
+});
+```
+We can use compile time expressions to generate a template and then recompile it as expression. Refer these examples [simple compilation](test/scenarios/compile_time_expressions/template.jt) and [complex compilation](test/scenarios/compile_time_expressions/two_level_path_processing.jt) for more details.
+
+### Comments
+Supports both c style single line (`//`) and block comments (`/* .. */`).
+Refer this [example](test/scenarios/comments/template.jt) for more details.
+
 For more examples, refer [Scenarios](test/scenarios)
@@ -74,4 +296,5 @@
 ```ts
-const engine = new JsonTemplateEngine(`'Hello ' + .name`);
-engine.evaluate({name: 'World'});
+const { JsonTemplateEngine } = require('rudder-json-template-engine');
+const engine = JsonTemplateEngine.create(`'Hello ' + .name`);
+engine.evaluate({name: 'World'}); // => 'Hello World'
 ```
@@ -78,0 +301,0 @@

No alert changes

Improved metrics

Total package byte prevSize

126856

increased by5.71%

Lines of code

2978

increased by0.07%

Number of lines in readme file

309

increased by259.3%

No dependency changes