
Security News
Deno 2.2 Improves Dependency Management and Expands Node.js Compatibility
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
An elegant armor-plated JavaScript lexer modelled after flex. Easily extensible to tailor to your need for perfection.
An elegant armor-plated JavaScript lexer modelled after flex. Easily extensible to tailor to your need for perfection.
Lexer may be installed on node.js via the node package manager using the command npm install lex
.
You may also install it on RingoJS using the command ringo-admin install aaditmshah/lexer
.
Creating a lexer is as simple as instantiating the constructor Lexer
.
var lexer = new Lexer;
After creating a lexer you may add rules to the lexer using the method addRule
. The first argument to the function must be a RegExp
object (the pattern to match). The second argument must be a function (the action to call when the pattern matches some text). The arguments passed to the function are the lexeme that was matched and all the substrings matched by parentheticals in the regular expression if any.
var lexer = new Lexer;
lexer.addRule(/[a-f\d]+/i, function (lexeme) {
return "HEX";
});
After adding rules to the lexer you may set the property input
of the lexer to any string that you wish to tokenize and then call the method lex
. The function returns the first non undefined
value returned by an action. Else it returns undefined
if it scans the entire input string. On calling lex
it starts scanning where it last left off. The addRule
and setInput
methods of the lexer support chaining.
var lines = 0;
var chars = 0;
(new Lexer).addRule(/\n/, function () {
lines++;
chars++;
}).addRule(/./, function () {
chars++;
}).setInput("Hello World!").lex();
If the lexer can't match any pattern then it executes the default rule which matches the next character in the input string. The default action may be specified as an argument to the constructor. Setting the property reject
on the this
object in an action to true
tells the lexer to reject the current rule and match the next best rule.
var row = 1;
var col = 1;
var lexer = new Lexer(function (char) {
throw new Error("Unexpected character at row " + row + ", col " + col + ": " + char);
});
lexer.addRule(/\n/, function () {
row++;
col = 1;
}, []);
lexer.addRule(/./, function () {
this.reject = true;
col++;
}, []);
lexer.input = "Hello World!";
lexer.lex();
You may even specify start conditions for every rule as an optional third argument to the addRule
method (which must be an array of unsigned integers). By default all rules are active in the initial state (i.e. 0
). Odd start conditions are inclusive while even start conditions are exclusive. Rules with an empty array as the third argument are always active.
The generated lexer may be used as a custom scanner for Jison. Actions must return tokens and associated text must be made available in the property yytext
on the object this
from within the action.
var Parser = require("jison").Parser;
var Lexer = require("lex");
var grammar = {
"bnf": {
"expression" :[[ "e EOF", "return $1;" ]],
"e" :[[ "NUMBER", "$$ = Number(yytext);" ]]
}
};
var parser = new Parser(grammar);
var lexer = parser.lexer = new Lexer;
lexer.addRule(/\s+/, function () {});
lexer.addRule(/[0-9]+(?:\.[0-9]+)?\b/, function (lexeme) {
this.yytext = lexeme;
return "NUMBER";
});
lexer.addRule(/$/, function () {
return "EOF";
});
parser.parse("2");
FAQs
An elegant armor-plated JavaScript lexer modelled after flex. Easily extensible to tailor to your need for perfection.
The npm package lex receives a total of 47,224 weekly downloads. As such, lex popularity was classified as popular.
We found that lex demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
Security News
React's CRA deprecation announcement sparked community criticism over framework recommendations, leading to quick updates acknowledging build tools like Vite as valid alternatives.
Security News
Ransomware payment rates hit an all-time low in 2024 as law enforcement crackdowns, stronger defenses, and shifting policies make attacks riskier and less profitable.