rudder-json-template-engine

The Customer Data Platform for Developers

Website · Documentation · Community Slack

---

rudder-json-template-engine

Motivation

We are an integration platform and support 200+ integrations. We implemented these integrations using native Javascript code to transform incoming events to destination payload, so in summary, it is JSON data manipulation. Maintaining all these integrations is challenging, so we explored jsonata to write less code to transform JSON data. While this library is excellent, we still need to meet our performance needs. For example, JSONata parses the given template, creates an Abstract Syntax Tree (AST), and interprets the AST for the given input. Since we need to traverse the AST every time, it is slow, so we wanted to build a template engine that generates Javascript code so there will be less overhead during the runtime.

Overview

A library to process JSON data using a custom syntax based on javascript and jspath. We thank the jspath authors for their excellent work, as our library is an extension of the original library. We also want to thank IBM team for their work on jsonata, as we have taken several ideas from the library. You can also consider our library as an alternative to jsonata.

This library generates a javascript function code from the template and then uses the function to evaluate the JSON data. It outputs the javascript code in the following stages:

1. Lexing (Tokenization)
2. Parsing (AST Creation)
3. Translation (Code generation)

  flowchart TD;
      A[Code] --> B[Convert code to tokens];
      B --> C[Parse tokens to create Expressions];
      C --> D[Combine expressions to create statements];
      D --> E[Combine statements to create AST];
      E --> F[Translate AST to JS code]

Engine class abstracts the above steps and provides a convenient way to use the json templates to evaluate the inputs.

Features

1. Variables
2. Arrays
3. Objects
4. Functions
5. Bindings
6. Comments
7. Paths
   • Filters
   • Selectors
   • Context variables
   • Simple paths
   • Rich paths
   • Paths options
8. Conditions
   • Comparisons
9. Math operations
10. Logical operations
11. Compile time expressions

For more examples, refer Scenarios

Syntax

Getting started

npm install rudder-json-template-engine

const engine = new JsonTemplateEngine(`'Hello ' + .name`);
engine.evaluate({name: 'World'});

Testing

npm test

Contribute

We would love to see you contribute to RudderStack. Get more information on how to contribute here.

License

The RudderStack rudder-json-template-engine is released under the MIT License.

Changelog

0.5.0 (2023-01-19)

Features

• add stryker mutation testing integration (#25) (55f8c22)