@lanetix/formula-fields-parser

NOTE TO FUTURE ENGINEER

Dear Future Engineer, Work was done on this package to upgrade Chevrotain from v0.23.0 to 4.0.0 so the package could be built with webpack and used in the UI. This was a non-trivial effort, and to the best of this engineers knowledge there is partity in the functioning of the parser as all tests pass. However, this upgrade was not published and is not used in the Lanetix code base. Keep this in mind when making future changes to this package -- if the goal is to extend the exisiting package and remain on chevrotain@0.23.0 then the commits related to the upgrade will need to be reverted first. -- December 2018

formula-fields-parser

This repository contains the WIP Formula Fields Parser. Currently implemented:

1. Lexer
2. Parser
3. Visitor
4. Errors
5. Symbol Table
6. Semantic Analysis
7. Evaluator

Exports

• Parser - The formula fields parser

• visit - The formula fields AST Visitor

• errors - Built in errors for use with the visitor

• withPostProcessors - A function to add post-processors to the parse and validate logic, similar to redux middleware. See the detailed explanation for more information

• compilers - An object of all the supported compilers. See the explanations for more information

  • evaluator - Compiles formula text to a javascript function that evaluates the formula when given a Record object
    • fromAst - Compiles a formula AST to a javascript function that evaluates the formula when given a Record object
    • withMiddleware - A function that takes post processors and returns an evaluator compiler. The post processors are applied after parsing the formula text, and before generating the evaluator.

• createEvaluatorFromAst - A function that takes an AST and returns a javascript function which will evaluate the formula when given a record object. See the detailed explanation for more information

• createEvaluatorCompiler - A function that takes middleware (if any) and returns a function that returns a result with an evaluator property. See the detailed explanation for more information.

default

The default export is a convenience function that takes the text of a formula, parses it, and runs semantic validation. It takes the following parameters:

• formulaText - The formula text
• options - An object containing options for parsing. Note that this object is also passed to all post-processors, so it can contain additional options if needed.
  • parseOptions - An object containing the options to give to the parser
  • recordType - The type schema for the record this formula is bound to. If provided, it will be used during validation to ensure all fields referenced in the formula exist on the given type.
  • optimizations - An object containing flags turning optimizations on or off
    • constantFold - Turns off constant folding if set to false

Result

It returns an object with the following properties:

• input - The formula text
• ast - The formula abstract syntax tree. This may be undefined if there are errors during the lexing or parsing phases.
• errors - An array of any lex, parse, or semantic errors that occurred when validating the formula. The array will exist, but be empty if no errors ocurred.

All errors are guaranteed to have the following properties, although some may have additional:

• name - The name of the error type (e.g. MismatchedTokenException)
• message - The error message
• location - The location where the error occurred

Parse Example

import parse from '@lanetix/formula-fields-parser'

// fields *are not* validated
const { ast, errors } = parse('CONCAT($first_name, $last_name)')

// fields are validated
const recordType = ...     // get from records service if not available
const { ast, errors } = parse('CONCAT($first_name, $last_name)', { recordType })

// Turn off constant folding
const { ast, errors } = parse('CONCAT("Hello ", "World")', { optimizations: { constantFold: false } })

// Only parse string literals
import parse, { Parser } from '@lanetix/formula-fields-parser'
const { ast, errors } = parse('"Hello World"', { parseOptions: { rule: Parser.stringLiteral } })

withPostProcessors

In addition to the normal parse -> optimize -> validate pipeline provided by the default export, additional post processors can be specified. Post- processors are extremely similar to Redux middleware, and it is advised to be familiar with such before continuing.

Post-Processor Explanation

Post processors have the following signature:

next => (parseResult, options) => { /* Implementation Logic */ }

• next - The next post processor in the chain. If not called, the post- processing chain will be aborted here. It must be called with parseResult and options, although the implementation logic is free to modify both objects

• parseResult - The parse result. Additional properties may be added, but ast and errors must always conform to the parse result standard format

• options - The options object given. If none are given, this will be an empty object, rather than undefined.

withPostProcessors Examples

Logs out the ast when the logAst option is set to true

import { withPostProcessors } from '@lanetix/formula-fields-parser'

const logAst = next => (parseResult, options) => {
  if (options.logAst) {
    console.log(parseResult.ast)
  }

  return next(parseResult, options)
}

const parse = withPostProcessors(logAst)

// ast is *not* logged by the logAst post-processor, because no options are given
const { ast, errors } = parse('CONCAT("Hello ", $name)')

// ast is logged by the logAst post-processor
const { ast, errors } = parse('CONCAT("Hello ", $name)', { logAst: true })

// can be combined with built in options
const options = {
  logAst: true,
  optimizations: {
    constantFold: false
  }
}
const { ast, errors } = parse('CONCAT("Hello ", "World")', options)

Conditionally return either the ast or the errors

import { withPostProcessors } from '@lanetix/formula-fields-parser'

// Note that next is not called, so this *must* be the last argument to
// withPostProcessors. If it isn't, post-processors after it will be completely
// ignored.
const asResult = next => (parseResult, options) => {
  if (parseResult.errors.length > 0) {
    return { tag: 'error', value: errors }
  } else {
    return { tag: 'ok', value: ast }
  }
}

const parse = withPostProcessors(asResult)

// Result is an ast
const result = parse('CONCAT("Hello ", $name)')

// Result is an array of errors
const result = parse('CONCAT(')

compilers

Evaluator

Takes a formula string and returns a result object with an extra evaluator property, which is an evaluator function. The evaluator function takes a single parameter, the record object, and returns the value of the formula when run on that record.

The evaluator will be undefined if any errors occurred during parsing or semantic validation of the formula. This can be checked via the errors object of the result.

NOTE: Accessing related fields (e.g. =$related.name) is not currently supported in the evaluator

fromAst

Takes an AST and only returns the evaluator function. It is assumed the AST given has been run through semantic validation. If not, unexpected runtime errors may be thrown when creating the evaluator, as well as during evaluation. Any AST created with the Parser will have passed these validations.

withMiddleware

Takes one or more post processors and returns an evaluator compiler function. The function works exactly like the compilers.evaluator function, only with the post processors acting like middleware. They will be run after the formula is parsed, but before the AST is compiled into an evaluator.

Examples

Evaluate a formula

import { compilers } from '@lanetix/formula-fields-parser'

const formulaText = '=50 / $field'
const { evaluator } = compilers.evaluator(formulaText)

evaluator({ field: 10 })    // returns 5
evaluator({ field: null })  // returns null

// Evaluators can throw exceptions for runtime errors:
try {
  evaluator({ field: 0 })
} catch (e) {
  // Catches an UnrepresentableNumberError, due to trying to divide by zero
}

// You can also provide options to the compiler
const options = { optimizations: { constantFold: false } }
const { evaluator: unoptimizedEvaluator } = compilers.evaluator(formulaText, options)
unoptimizedEvaluator({ field: 10 })  // still 5

Evaluate from an AST

import { compilers } from '@lanetix/formula-fields-parser'

const ast = ... // some ast from the parser
const evaluator = compilers.evaluator.fromAst(ast)

const value = evaluator({ field: 'something' })

Evaluate a formula with custom middleware

import { compilers } from '@lanetix/formula-fields-parser'

const debugMiddleware = (next) => (parseResult, options) => {
  console.log(parseResult)
  return next(parseResult, options)
}

const myEvaluatorCompiler = compilers.evaluator.withMiddleware(debugMiddleware)

const formulaText = '=50 / $field'

// Runs the debugMiddleware before generating the evaluator
const { evaluator } = myEvaluatorCompiler(formulaText)

evaluator({ field: 10 }) // returns 5

// You can also provide options to the compiler
const options = { optimizations: { constantFold: false } }
const unoptimizedEvaluator = myEvaluatorCompiler(formulaText, options)
unoptimizedEvaluator({ field: 10 }) // still 5