chevrotain

What is chevrotain?

Chevrotain is a fast and feature-rich parser building toolkit for JavaScript. It can be used to build parsers for DSLs, programming languages, data formats, and more. It provides a set of APIs for defining grammar rules and constructing a parser based on those rules.

What are chevrotain's main functionalities?

Defining Token Types

This code sample demonstrates how to define token types using Chevrotain. Tokens are the basic building blocks of the syntax for a language or format. In this example, we define tokens for integers and the plus and minus symbols.

const { createToken, Lexer } = require('chevrotain');

const Integer = createToken({ name: 'Integer', pattern: /\d+/ });
const Plus = createToken({ name: 'Plus', pattern: /\+/ });
const Minus = createToken({ name: 'Minus', pattern: /-/ });

const allTokens = [Plus, Minus, Integer];
const MyLexer = new Lexer(allTokens);

Building a Parser

This code sample shows how to build a parser using Chevrotain. The parser is defined as a class that extends `CstParser` and uses rules to define the grammar of the language. In this example, we define a simple grammar for addition expressions.

const { CstParser } = require('chevrotain');

class MyParser extends CstParser {
  constructor() {
    super(allTokens);
    this.RULE('expression', () => {
      this.SUBRULE(this.additionExpression);
    });

    this.RULE('additionExpression', () => {
      this.CONSUME(Integer);
      this.MANY(() => {
        this.OR([
          { ALT: () => { this.CONSUME(Plus); this.CONSUME2(Integer); } },
          { ALT: () => { this.CONSUME(Minus); this.CONSUME2(Integer); } }
        ]);
      });
    });

    this.performSelfAnalysis();
  }
}

const parser = new MyParser();

Parsing Text

This code sample illustrates how to parse text using a lexer and parser defined with Chevrotain. The text is tokenized by the lexer, and then the tokens are fed into the parser to produce a Concrete Syntax Tree (CST), which can be used for further processing such as interpretation or transformation into an Abstract Syntax Tree (AST).

const { tokenMatcher } = require('chevrotain');

const text = '1 + 2 - 3';
const lexingResult = MyLexer.tokenize(text);

if (lexingResult.errors.length === 0) {
  parser.input = lexingResult.tokens;
  const cst = parser.expression();

  if (parser.errors.length === 0) {
    // cst can now be used to create an AST or for interpretation.
  } else {
    // parser errors are present
  }
} else {
  // lexing errors are present
}

Other packages similar to chevrotain

pegjs

PEG.js is a simple parser generator for JavaScript that produces fast parsers with excellent error reporting. It uses Parsing Expression Grammars (PEG) as the input. Compared to Chevrotain, PEG.js has a different approach to defining grammars (PEG vs. Chevrotain's API) and does not require manual token definition.

antlr4

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator that can be used to read, process, execute, or translate structured text or binary files. It's widely used to build languages, tools, and frameworks. ANTLR4 has a Java-based toolchain with targets for multiple languages including JavaScript. It is more complex than Chevrotain and has a steeper learning curve, but it is also more feature-rich.

jison

Jison is an API for creating parsers in JavaScript that works similarly to yacc. It takes a context-free grammar as input and outputs a JavaScript file capable of parsing the language described by that grammar. Jison handles both lexical and syntactical analysis, which means it combines the features of both lexer and parser generators. It is less modular than Chevrotain but can be easier to use for those familiar with yacc or bison.

nearley

Nearley is a simple, fast, and powerful parsing toolkit for JavaScript. It is based on Earley's algorithm, which is suitable for parsing complex and ambiguous grammars. Nearley is designed to be more user-friendly and flexible than traditional parser generators. It allows for dynamic grammar and can handle any kind of parsing task. It is comparable to Chevrotain in terms of ease of use but uses a different underlying algorithm for parsing.

README

Chevrotain

Introduction

Chevrotain is a blazing fast and feature rich Parser Building Toolkit for JavaScript. It can be used to build parsers/compilers/interpreters for various use cases ranging from simple configuration files, to full fledged programing languages.

A more in depth description of Chevrotain can be found in this great article on: Parsing in JavaScript: Tools and Libraries.

It is important to note that Chevrotain is NOT a parser generator. It solves the same kind of problems as a parser generator, just without any code generation. Chevrotain Grammars are pure code which can be created/debugged/edited as any other pure code without requiring any new tools or processes.

TLDR

• Online Playground
• Getting Started Tutorial
• Performance benchmark

Installation

• npm: npm install chevrotain
• Browser: The npm package contains Chevrotain as concatenated and minified files ready for use in a browser. These can also be accessed directly via UNPKG in a script tag.
  • Latest:
    • https://unpkg.com/chevrotain/lib/chevrotain.js
    • https://unpkg.com/chevrotain/lib/chevrotain.min.js
  • Explicit version number:
    • https://unpkg.com/chevrotain@4.1.0/lib/chevrotain.js
    • https://unpkg.com/chevrotain@4.1.0/lib/chevrotain.min.js

Documentation & Resources

• Getting Started Tutorial.

• Sample Grammars.

• FAQ.

• Other Examples.

• HTML API docs.

  • The Parsing DSL Docs.

Dependencies

There is a single dependency to regexp-to-ast library. This dependency is included in the bundled artifacts, for ease of consumption in browsers.

Compatibility

Chevrotain runs on any modern JavaScript ES5.1 runtime.

• The CI build runs the tests under:

  • Node.js (6 / 8 / 10).
  • Latest stable: Chrome, FireFox, Safari, IE Edge and IE 11.

• Uses UMD to work with common module loaders (browser global / amd / commonjs).

Contributions

Contributions are greatly appreciated. See CONTRIBUTING.md for details.

Where used

Some interesting samples:

• JHipster Domain Language
• Metabase BI expression parser.
• Eve Programing Language.
• BioModelAnalyzer's ChatBot parser.
• Bombadil Toml Parser
• BrightScript Parser