Version 3.0.1 (2017-01-30)

• Fixed: ES2015 unicode escapes with more than 6 hex digits are now matched correctly.

Version 3.0.0 (2017-01-11)

This release contains one breaking change, that should [improve performance in V8][v8-perf]:

So how can you, as a JavaScript developer, ensure that your RegExps are fast? If you are not interested in hooking into RegExp internals, make sure that neither the RegExp instance, nor its prototype is modified in order to get the best performance:

var re = /./g;
re.exec(""); // Fast path.
re.new_property = "slow";

This module used to export a single regex, with .matchToToken bolted on, just like in the above example. This release changes the exports of the module to avoid this issue.

Before:

import jsTokens from "js-tokens";
// or:
var jsTokens = require("js-tokens");
var matchToToken = jsTokens.matchToToken;

After:

import jsTokens, { matchToToken } from "js-tokens";
// or:
var jsTokens = require("js-tokens").default;
var matchToToken = require("js-tokens").matchToToken;

Version 2.0.0 (2016-06-19)

• Added: Support for ES2016. In other words, support for the ** exponentiation operator.

These are the breaking changes:

• '**'.match(jsTokens) no longer returns ['*', '*'], but ['**'].
• '**='.match(jsTokens) no longer returns ['*', '*='], but ['**='].

Version 1.0.3 (2016-03-27)

• Improved: Made the regex ever so slightly smaller.
• Updated: The readme.

Version 1.0.2 (2015-10-18)

• Improved: Limited npm package contents for a smaller download. Thanks to @zertosh!

Version 1.0.1 (2015-06-20)

• Fixed: Declared an undeclared variable.

Version 1.0.0 (2015-02-26)

• Changed: Merged the 'operator' and 'punctuation' types into 'punctuator'. That type is now equivalent to the Punctuator token in the ECMAScript specification. (Backwards-incompatible change.)
• Fixed: A - followed by a number is now correctly matched as a punctuator followed by a number. It used to be matched as just a number, but there is no such thing as negative number literals. (Possibly backwards-incompatible change.)

Version 0.4.1 (2015-02-21)

• Added: Support for the regex u flag.

Version 0.4.0 (2015-02-21)

• Improved: jsTokens.matchToToken performance.
• Added: Support for octal and binary number literals.
• Added: Support for template strings.

Version 0.3.1 (2015-01-06)

• Fixed: Support for unicode spaces. They used to be allowed in names (which is very confusing), and some unicode newlines were wrongly allowed in strings and regexes.