json-cst

json-cst

This package parses a JSON into CST (Concrete Syntax Tree), similar to an AST but more low-level and with ties to the lexer tokens. It uses json-lexer to parse the file into tokens. The speed is practically the same as json-to-ast (it's ~10% faster than json-to-ast), but it's far smaller (even including json-lexer).

Install

npm i json-cst or yarn add json-cst

This is a pure ESM package, and requires Node.js >=14.13.1

Simple usage

Exports

The package exports parse(json: string, options: ParseCstOptions): CstNode.

options is an optional object which can contain includeValueTokens: true to include the value tokens in the result, meaning, for objects and arrays, they will include the slice of tokens for the beginning and end of the object/array.

Definition

The tokens are parsed into a hierarchy of nodes, each with a "kind" property:

type CstKindLiteral = 'literal'; // null, true, false
type CstKindNumber = 'number';
type CstKindString = 'string';
type CstKindObjectPropertyColon = 'object-property-colon';
type CstKindObjectProperty = 'object-property';
type CstKindObject = 'object';
type CstKindArrayElement = 'array-element';
type CstKindArray = 'array';

And the CstNode returned by parse() is a CstValueNode, i.e. one of:

• CstNodeLiteral
• CstNodeNumber
• CstNodeString
• CstNodeObject
• CstNodeArray

Other nodes are:

• CstNodeObjectProperty
• CstNodeObjectPropertyColon
• CstNodeArrayElement

Each token contain a { range: CstTokenRange } where

interface CstTokenRange {
    start: number;
	end: number;
}

Each of the primitive tokens CstNodeLiteral, CstNodeNumber and CstNodeString contain { token: Token } being the raw token from json-lexer.

Object and array tokens CstNodeObject and CstNodeArray contain a property children being an array of either CstNodeObjectProperty or CstNodeArrayElement.

A CstNodeObjectProperty has a keyToken property being the lexer token for the property name, and a valueNode being a CstNode. A CstNodeArrayElement also has a valueNode.

See types.ts for exact typings.