snapdragon

What is snapdragon?

The snapdragon npm package is a highly pluggable, low-level compiler for creating compilers and other text processing applications. It provides a robust framework for parsing, compiling, and rendering custom grammars in a structured way.

What are snapdragon's main functionalities?

Parsing

This feature allows you to define custom parsing rules. The code sample demonstrates how to create a simple text parser that recognizes sequences of lowercase letters.

const Snapdragon = require('snapdragon');
const snapdragon = new Snapdragon();
snapdragon.parser.use(function() {
  this.set('text', function() {
    const pos = this.position();
    const m = this.match(/^[a-z]+/);
    if (!m) return;
    return pos(this.node({type: 'text', val: m[0]}));
  });
});
const ast = snapdragon.parse('hello');
console.log(ast);

Compiling

This feature allows you to compile parsed input into a new form. The code sample shows how to compile a parsed abstract syntax tree (AST) into uppercase text.

const Snapdragon = require('snapdragon');
const snapdragon = new Snapdragon();
snapdragon.compiler.use(function(node) {
  if (node.type === 'text') {
    return node.val.toUpperCase();
  }
});
const ast = snapdragon.parse('hello');
const result = snapdragon.compile(ast).output;
console.log(result);

Rendering

This feature involves rendering the compiled output. The code sample illustrates how to render the output of a parsed AST, adding a space after each text node.

const Snapdragon = require('snapdragon');
const snapdragon = new Snapdragon();
snapdragon.renderer.use(function(node) {
  if (node.type === 'text') {
    this.emit(node.val + ' '); // add space after each text node
  }
});
const ast = snapdragon.parse('hello world');
const result = snapdragon.render(ast);
console.log(result);

Other packages similar to snapdragon

chevrotain

Chevrotain is a JavaScript parser building toolkit which is similar to snapdragon but focuses more on performance and ease of use for creating complex parsers without writing too much boilerplate code. It differs from snapdragon in that it provides a richer set of built-in features for defining grammars and performing lexical analysis.

nearley

Nearley is another parser toolkit that uses a different approach by leveraging grammar files written in a special format. It is similar to snapdragon in that it allows for the creation of custom parsers, but it automates more of the process and can handle more complex, ambiguous grammars.

pegjs

PEG.js is a simple parser generator that allows you to create parsers directly from a string representation of a Parsing Expression Grammar. It is similar to snapdragon in its purpose but differs in that it generates parsers from a high-level description rather than requiring manual definition of parsing behavior.

README

snapdragon

Easy-to-use plugin system for creating powerful, fast and versatile parsers and compilers, with built-in source-map support.

Please consider following this project's author, Jon Schlinkert, and consider starring the project to show your :heart: and support.

Table of Contents

Details

• Install
• Quickstart example
• Parsing
• Compiling
• All together
• API
  • Parse
  • Compile
• Snapdragon in the wild
• History
  • v0.9.0
  • v0.5.0
• About

Install

Install with npm:

$ npm install --save snapdragon

Created by jonschlinkert and doowb.

Features

• Bootstrap your own parser, get sourcemap support for free
• All parsing and compiling is handled by simple, reusable middleware functions
• Inspired by the parsers in pug and css.

Quickstart example

All of the examples in this document assume the following two lines of setup code exist first:

var Snapdragon = require('snapdragon');
var snapdragon = new Snapdragon();

Parse a string

var ast = snapdragon.parser
  // parser handlers (essentially middleware)
  // used for parsing substrings to create tokens
  .set('foo', function () {})
  .set('bar', function () {})
  .parse('some string', options);

Compile an AST returned from .parse()

var result = snapdragon.compiler
  // compiler handlers (essentially middleware), 
  // called on a node when the `node.type` matches
  // the name of the handler
  .set('foo', function () {})
  .set('bar', function () {})
  // pass the `ast` from the parse method
  .compile(ast)

// the compiled string
console.log(result.output);

See the examples.

Parsing

Parser handlers

Parser handlers are middleware functions responsible for matching substrings to create tokens:

Example handler

var ast = snapdragon.parser
  .set('dot', function() {
    var pos = this.position();
    var m = this.match(/^\./);
    if (!m) return;
    return pos({
      // the "type" will be used by the compiler later on,
      // we'll go over this in the compiler docs
      type: 'dot',
      // "val" is the string captured by ".match",
      // in this case that would be '.'
      val: m[0]
    });
  })
  .parse('.'[, options])

As a side node, it's not scrictly required to set the type on the token, since the parser will add it to the token if it's undefined, based on the name of the handler. But it's good practice since tokens aren't always returned.

Example token

And the resulting tokens look something like this:

{ 
  type: 'dot',
  val: '.' 
}

Position

Next, pos() is called on the token as it's returned, which patches the token with the position of the string that was captured:

{ type: 'dot',
  val: '.',
  position:
   { start: { lineno: 1, column: 1 },
     end: { lineno: 1, column: 2 } }}

Life as an AST node

When the token is returned, the parser pushes it onto the nodes array of the "previous" node (since we're in a tree, the "previous" node might be literally the last node that was created, or it might be the "parent" node inside a nested context, like when parsing brackets or something with an open or close), at which point the token begins its life as an AST node.

Wrapping up

In the parser calls all handlers and cannot find a match for a substring, an error is thrown.

Assuming the parser finished parsing the entire string, an AST is returned.

Compiling

The compiler's job is to take the AST created by the parser and convert it to a new string. It does this by iterating over each node on the AST and calling a function on the node based on its type.

This function is called a "handler".

Compiler handlers

Handlers are named middleware functions that are called on a node when node.type matches the name of a registered handler.

var result = snapdragon.compiler
  .set('dot', function (node) {
    console.log(node.val)
    //=> '.'
    return this.emit(node.val);
  })

If node.type does not match a registered handler, an error is thrown.

Source maps

If you want source map support, make sure to emit the entire node as the second argument as well (this allows the compiler to get the node.position).

var res = snapdragon.compiler
  .set('dot', function (node) {
    return this.emit(node.val, node);
  })

All together

This is a very basic example, but it shows how to parse a dot, then compile it as an escaped dot.

var Snapdragon = require('..');
var snapdragon = new Snapdragon();

var ast = snapdragon.parser
  .set('dot', function () {
    var pos = this.position();
    var m = this.match(/^\./);
    if (!m) return;
    return pos({
      type: 'dot',
      val: m[0]
    })
  })
  .parse('.')

var result = snapdragon.compiler
  .set('dot', function (node) {
    return this.emit('\\' + node.val);
  })
  .compile(ast)

console.log(result.output);
//=> '\.'

API

Parser

Create a new Parser with the given input and options.

Params

• input {String}
• options {Object}

Example

var Snapdragon = require('snapdragon');
var Parser = Snapdragon.Parser;
var parser = new Parser();

.error

Throw a formatted error message with details including the cursor position.

Params

• msg {String}: Message to use in the Error.
• node {Object}
• returns {undefined}

Example

parser.set('foo', function(node) {
  if (node.val !== 'foo') {
    throw this.error('expected node.val to be "foo"', node);
  }
});

.define

Define a non-enumberable property on the Parser instance. This is useful in plugins, for exposing methods inside handlers.

Params

• key {String}: propery name
• val {any}: property value
• returns {Object}: Returns the Parser instance for chaining.

Example

parser.define('foo', 'bar');

.node

Create a new Node with the given val and type.

Params

• val {Object}
• type {String}
• returns {Object}: returns the Node instance.

Example

parser.node('/', 'slash');

.position

Mark position and patch node.position.

• returns {Function}: Returns a function that takes a node

Example

parser.set('foo', function(node) {
  var pos = this.position();
  var match = this.match(/foo/);
  if (match) {
    // call `pos` with the node
    return pos(this.node(match[0]));
  }
});

.set

Add parser type with the given visitor fn.

Params

• type {String}
• fn {Function}

Example

 parser.set('all', function() {
   var match = this.match(/^./);
   if (match) {
     return this.node(match[0]);
   }
 });

.get

Get parser type.

Params

• type {String}

Example

var fn = parser.get('slash');

.push

Push a node onto the stack for the given type.

Params

• type {String}
• returns {Object} token

Example

parser.set('all', function() {
  var match = this.match(/^./);
  if (match) {
    var node = this.node(match[0]);
    this.push(node);
    return node;
  }
});

.pop

Pop a token off of the stack of the given type.

Params

• type {String}
• returns {Object}: Returns a token

Example

parser.set('close', function() {
  var match = this.match(/^\}/);
  if (match) {
    var node = this.node({
      type: 'close',
      val: match[0]
    });

    this.pop(node.type);
    return node;
  }
});

.isInside

Return true if inside a "set" of the given type. Sets are created manually by adding a type to parser.sets. A node is "inside" a set when an *.open node for the given type was previously pushed onto the set. The type is removed from the set by popping it off when the *.close node for the given type is reached.

Params

• type {String}
• returns {Boolean}

Example

parser.set('close', function() {
  var pos = this.position();
  var m = this.match(/^\}/);
  if (!m) return;
  if (!this.isInside('bracket')) {
    throw new Error('missing opening bracket');
  }
});

.isType

Return true if node is the given type.

Params

• node {Object}
• type {String}
• returns {Boolean}

Example

parser.isType(node, 'brace');

.prev

Get the previous AST node from the parser.stack (when inside a nested context) or parser.nodes.

• returns {Object}

Example

var prev = this.prev();

.prev

Match regex, return captures, and update the cursor position by match[0] length.

Params

• regex {RegExp}
• returns {Object}

Example

// make sure to use the starting regex boundary: "^"
var match = this.match(/^\./);

Params

• input {String}
• returns {Object}: Returns an AST with ast.nodes

Example

var ast = parser.parse('foo/bar');

Compiler

Create a new Compiler with the given options.

Params

• options {Object}
• state {Object}: Optionally pass a "state" object to use inside visitor functions.

Example

var Snapdragon = require('snapdragon');
var Compiler = Snapdragon.Compiler;
var compiler = new Compiler();

.error

Throw a formatted error message with details including the cursor position.

Params

• msg {String}: Message to use in the Error.
• node {Object}
• returns {undefined}

Example

compiler.set('foo', function(node) {
  if (node.val !== 'foo') {
    throw this.error('expected node.val to be "foo"', node);
  }
});

.emit

Concat the given string to compiler.output.

Params

• string {String}
• node {Object}: Optionally pass the node to use for position if source maps are enabled.
• returns {String}: returns the string

Example

compiler.set('foo', function(node) {
  this.emit(node.val, node);
});

.noop

Emit an empty string to effectively "skip" the string for the given node, but still emit the position and node type.

Params

• {Object}: node

Example

// example: do nothing for beginning-of-string
snapdragon.compiler.set('bos', compiler.noop);

.define

Define a non-enumberable property on the Compiler instance. This is useful in plugins, for exposing methods inside handlers.

Params

• key {String}: propery name
• val {any}: property value
• returns {Object}: Returns the Compiler instance for chaining.

Example

compiler.define('customMethod', function() {
  // do stuff
});

.set

Add a compiler fn for the given type. Compilers are called when the .compile method encounters a node of the given type to generate the output string.

Params

• type {String}
• fn {Function}

Example

compiler
  .set('comma', function(node) {
    this.emit(',');
  })
  .set('dot', function(node) {
    this.emit('.');
  })
  .set('slash', function(node) {
    this.emit('/');
  });

.get

Get the compiler of the given type.

Params

• type {String}

Example

var fn = compiler.get('slash');

.visit

Visit node using the registered compiler function associated with the node.type.

Params

• node {Object}
• returns {Object}: returns the node

Example

compiler
  .set('i', function(node) {
    this.visit(node);
  })

.mapVisit

Iterate over node.nodes, calling visit on each node.

Params

• node {Object}
• returns {Object}: returns the node

Example

compiler
  .set('i', function(node) {
    utils.mapVisit(node);
  })

.compile

Compile the given AST and return a string. Iterates over ast.nodes with mapVisit.

Params

• ast {Object}
• options {Object}: Compiler options
• returns {Object}: returns the node

Example

var ast = parser.parse('foo');
var str = compiler.compile(ast);

Snapdragon in the wild

A few of the libraries that use snapdragon:

• braces: Bash-like brace expansion, implemented in JavaScript. Safer than other brace expansion libs, with complete support… more | homepage
• breakdance: Breakdance is a node.js library for converting HTML to markdown. Highly pluggable, flexible and easy… more | homepage
• expand-brackets: Expand POSIX bracket expressions (character classes) in glob patterns. | homepage
• extglob: Extended glob support for JavaScript. Adds (almost) the expressive power of regular expressions to glob… more | homepage
• micromatch: Glob matching for javascript/node.js. A drop-in replacement and faster alternative to minimatch and multimatch. | homepage
• nanomatch: Fast, minimal glob matcher for node.js. Similar to micromatch, minimatch and multimatch, but complete Bash… more | homepage

History

v0.9.0

Breaking changes!

In an attempt to make snapdragon lighter, more versatile, and more pluggable, some major changes were made in this release.

• parser.capture was externalized to snapdragon-capture
• parser.capturePair was externalized to snapdragon-capture-set
• Nodes are now an instance of snapdragon-node

v0.5.0

Breaking changes!

Substantial breaking changes were made in v0.5.0! Most of these changes are part of a larger refactor that will be finished in 0.6.0, including the introduction of a Lexer class.

• Renderer was renamed to Compiler
• the .render method was renamed to .compile

About

Contributing

Pull requests and stars are always welcome. For bugs and feature requests, please create an issue.

Running Tests

Running and reviewing unit tests is a great way to get familiarized with a library and its API. You can install dependencies and run tests with the following command:

$ npm install && npm test

Building docs

(This project's readme.md is generated by verb, please don't edit the readme directly. Any changes to the readme must be made in the .verb.md readme template.)

To generate the readme, run the following command:

$ npm install -g verbose/verb#dev verb-generate-readme && verb

Related projects

A few of the libraries that use snapdragon:

• snapdragon-capture-set: Plugin that adds a .captureSet() method to snapdragon, for matching and capturing substrings that have… more | homepage
• snapdragon-capture: Snapdragon plugin that adds a capture method to the parser instance. | homepage
• snapdragon-node: Snapdragon utility for creating a new AST node in custom code, such as plugins. | homepage
• snapdragon-util: Utilities for the snapdragon parser/compiler. | homepage

Contributors

Commits	Contributor
156	jonschlinkert
3	doowb
2	danez
1	EdwardBetts

Author

Jon Schlinkert

• LinkedIn Profile
• GitHub Profile
• Twitter Profile

License

Copyright © 2018, Jon Schlinkert. Released under the MIT License.

---

This file was generated by verb-generate-readme, v0.6.0, on March 20, 2018.