regexp-tree

What is regexp-tree?

The regexp-tree npm package is a toolkit for working with regular expressions in JavaScript. It provides functionalities for parsing, transforming, and optimizing regular expressions, making it easier to manipulate and understand complex regex patterns.

What are regexp-tree's main functionalities?

Parsing

This feature allows you to parse a regular expression into an Abstract Syntax Tree (AST). The AST can then be used for further analysis or transformation.

const { parse } = require('regexp-tree');
const ast = parse('/a(b|c)*d/');
console.log(JSON.stringify(ast, null, 2));

Transforming

This feature allows you to transform a regular expression by applying custom transformations to its AST. In this example, the character 'a' is transformed to 'z'.

const { transform } = require('regexp-tree');
const transformed = transform('/a(b|c)*d/', {
  onChar(path) {
    if (path.node.value === 'a') {
      path.node.value = 'z';
    }
  }
});
console.log(transformed.getSource());

Optimizing

This feature allows you to optimize a regular expression by simplifying it. In this example, the non-capturing group '(?:b|c)' is optimized.

const { optimize } = require('regexp-tree');
const optimized = optimize('/a(?:b|c)d/');
console.log(optimized.getSource());

Other packages similar to regexp-tree

regexpp

regexpp is a regular expression parser for ECMAScript. It provides functionalities to parse regular expressions into an AST, similar to regexp-tree. However, it does not offer transformation or optimization features.

regexp-parser

regexp-parser is another package for parsing regular expressions into an AST. It is simpler and more lightweight compared to regexp-tree, but it lacks the transformation and optimization capabilities.

regexgen

regexgen generates regular expressions from a set of strings. While it does not parse or transform existing regex patterns, it focuses on creating new regex patterns that match a given set of strings, which is a different use case compared to regexp-tree.

README

regexp-tree

Regular expressions parser

Table of Contents

• Installation
• Development
• Usage as a CLI
• Usage from Node
• Capturing locations
• AST nodes specification

Installation

The parser can be installed as an NPM module:

npm install -g regexp-tree

regexp-tree --help

Development

1. Fork https://github.com/DmitrySoshnikov/regexp-tree repo
2. Make your changes
3. Make sure npm test still passes (add new tests if needed)
4. Submit a PR

The regexp-tree parser is implemented as an automatic LR parser using Syntax tool. The parser module is generated from the regexp grammar, which is based on the regular expressions grammar used in ECMAScript.

For development from the github repository, run build command to generate the parser module:

git clone https://github.com/<your-github-account>/regexp-tree.git
cd regexp-tree
npm install
npm run build

./bin/regexp-tree --help

NOTE: You need to run build command every time you change the grammar file.

Usage as a CLI

Check the options available from CLI:

regexp-tree --help

Usage: regexp-tree [options]

Options:
   -e, --expression   A regular expression to be parsed
   -l, --loc          Whether to capture AST node locations

To parse a regular expression, pass -e option:

regexp-tree -e '/a|b/i'

Which produces an AST node corresponding to this regular expression:

{
  "type": "RegExp",
  "body": {
    "type": "Disjunction",
    "left": {
      "type": "Char",
      "value": "a",
      "kind": "simple"
    },
    "right": {
      "type": "Char",
      "value": "b",
      "kind": "simple"
    }
  },
  "flags": [
    "i"
  ]
}

Usage from Node

The parser can also be used as a Node module:

const regexpTree = require('regexp-tree');

const regexpString = (/a|b/i).toString();

console.log(regexpTree.parse(regexpString)); // RegExp AST

Capturing locations

For source code transformation tools it might be useful also to capture locations of the AST nodes. From the command line it's controlled via the '-l' option:

regexp-tree -e '/ab/' -l

This attaches loc object to each AST node:

{
  "type": "RegExp",
  "body": {
    "type": "Alternative",
    "expressions": [
      {
        "type": "Char",
        "value": "a",
        "kind": "simple",
        "loc": {
          "start": 1,
          "end": 2
        }
      },
      {
        "type": "Char",
        "value": "b",
        "kind": "simple",
        "loc": {
          "start": 2,
          "end": 3
        }
      }
    ],
    "loc": {
      "start": 1,
      "end": 3
    }
  },
  "flags": [],
  "loc": {
    "start": 0,
    "end": 4
  }
}

From Node it's controlled via setOptions method exposed on the parser:

const regexpTree = require('regexp-tree');

const parsed = regexpTree
  .setOptions({captureLocations: true})
  .parse('/a|b/');

NOTE: the format of a regexp should be /Body/Flags. The flags can be optional.

AST nodes specification

Below are the AST node types corresponding to different regular expressions sub-patterns:

TODO