chevrotain

Defining Token Types

This code sample demonstrates how to define token types using Chevrotain. Tokens are the basic building blocks of the syntax for a language or format. In this example, we define tokens for integers and the plus and minus symbols.

const { createToken, Lexer } = require('chevrotain');

const Integer = createToken({ name: 'Integer', pattern: /\d+/ });
const Plus = createToken({ name: 'Plus', pattern: /\+/ });
const Minus = createToken({ name: 'Minus', pattern: /-/ });

const allTokens = [Plus, Minus, Integer];
const MyLexer = new Lexer(allTokens);

Building a Parser

This code sample shows how to build a parser using Chevrotain. The parser is defined as a class that extends `CstParser` and uses rules to define the grammar of the language. In this example, we define a simple grammar for addition expressions.

const { CstParser } = require('chevrotain');

class MyParser extends CstParser {
  constructor() {
    super(allTokens);
    this.RULE('expression', () => {
      this.SUBRULE(this.additionExpression);
    });

    this.RULE('additionExpression', () => {
      this.CONSUME(Integer);
      this.MANY(() => {
        this.OR([
          { ALT: () => { this.CONSUME(Plus); this.CONSUME2(Integer); } },
          { ALT: () => { this.CONSUME(Minus); this.CONSUME2(Integer); } }
        ]);
      });
    });

    this.performSelfAnalysis();
  }
}

const parser = new MyParser();

Parsing Text

This code sample illustrates how to parse text using a lexer and parser defined with Chevrotain. The text is tokenized by the lexer, and then the tokens are fed into the parser to produce a Concrete Syntax Tree (CST), which can be used for further processing such as interpretation or transformation into an Abstract Syntax Tree (AST).

const { tokenMatcher } = require('chevrotain');

const text = '1 + 2 - 3';
const lexingResult = MyLexer.tokenize(text);

if (lexingResult.errors.length === 0) {
  parser.input = lexingResult.tokens;
  const cst = parser.expression();

  if (parser.errors.length === 0) {
    // cst can now be used to create an AST or for interpretation.
  } else {
    // parser errors are present
  }
} else {
  // lexing errors are present
}