CJS Module Lexer
A very fast JS CommonJS module syntax lexer used to detect the most likely list of named exports of a CommonJS module.
Outputs the list of named exports (exports.name = ...
), whether the __esModule
interop flag is used, and possible module reexports (module.exports = require('...')
).
For an example of the performance, Angular 1 (720KiB) is fully parsed in 5ms, in comparison to the fastest JS parser, Acorn which takes over 100ms.
Comprehensively handles the JS language grammar while remaining small and fast. - ~10ms per MB of JS cold and ~5ms per MB of JS warm, see benchmarks for more info.
Usage
npm install cjs-module-lexer
For use in CommonJS:
const { init, parse } = require('cjs-module-lexer');
(async () => {
await init;
const { exports, reexports, esModule } = parse(`
// named exports detection
module.exports.a = 'a';
(function () {
exports.b = 'b';
})();
Object.defineProperty(exports, 'c', { value: 'c' });
/* exports.d = 'not detected'; */
// reexports detection
if (maybe) module.exports = require('./dep1.js');
if (another) module.exports = require('./dep2.js');
// __esModule detection
Object.defineProperty(module.exports, '__esModule', { value: true })
`);
})();
An ES module version is also available from dist/lexer.js
, automatically enabled via "exports
":
Supported
Only exports that are valid identifiers (as defined by ECMA-262) are returned.
This includes filtering out the strict reserved words only - implements
, interface
, let
, package
, private
, protected
, public
, static
, yield
, enum
.
- All
exports.a =
, exports['a'] =
and module.exports.a =
style assignments. - All
Object.defineProperty(module.exports, 'name'
or Object.defineProperty(exports, 'name'
assignments - All
module.exports = require('string')
assignments - Any instance of
__webpack_exports__, "name"
results in these webpack exports being returned only,
and __esModule
is inferred as an export.
Not Supported
- No scope analysis:
(function (exports) {
exports.a = 'a';
})(notExports);
(function (m) {
m.a = 'a';
})(exports);
module.exports
require assignment only handled at the base-level
module.exports = require('./a.js');
if (condition)
module.exports = require('./b.js');
if (condition) {
module.exports = require('./c.js');
}
(function () {
module.exports = require('./d.js');
})();
- No object parsing:
Object.defineProperties(exports, {
a: { value: 'a' },
b: { value: 'b' }
});
module.exports = {
c: 'c',
d: 'd'
}
- Webpack exports heuristic
exports.a = 'a';
exports.b = 'b';
__webpack_require__.d(__webpack_exports__, "WP_A", function() { return setBaseUrl; });
__webpack_require__.d(__webpack_exports__, "WP_B", function() { return setBaseUrl; });
Environment Support
Node.js 10+, and all browsers with Web Assembly support.
Grammar Support
- Token state parses all line comments, block comments, strings, template strings, blocks, parens and punctuators.
- Division operator / regex token ambiguity is handled via backtracking checks against punctuator prefixes, including closing brace or paren backtracking.
- Always correctly parses valid JS source, but may parse invalid JS source without errors.
Benchmarks
Benchmarks can be run with npm run bench
.
Current results:
Cold Run, All Samples
test/samples/*.js (3057 KiB)
> 24ms
Warm Runs (average of 25 runs)
test/samples/angular.js (719 KiB)
> 5.12ms
test/samples/angular.min.js (188 KiB)
> 3.04ms
test/samples/d3.js (491 KiB)
> 4.08ms
test/samples/d3.min.js (274 KiB)
> 2.04ms
test/samples/magic-string.js (34 KiB)
> 0ms
test/samples/magic-string.min.js (20 KiB)
> 0ms
test/samples/rollup.js (902 KiB)
> 5.92ms
test/samples/rollup.min.js (429 KiB)
> 3.08ms
Warm Runs, All Samples (average of 25 runs)
test/samples/*.js (3057 KiB)
> 17.4ms
Building
To build download the WASI SDK from https://github.com/CraneStation/wasi-sdk/releases.
The Makefile assumes that the clang
in PATH corresponds to LLVM 8 (provided by WASI SDK as well, or a standard clang 8 install can be used as well), and that ../wasi-sdk-6
contains the SDK as extracted above, which is important to locate the WASI sysroot.
The build through the Makefile is then run via make lib/lexer.wasm
, which can also be triggered via npm run build-wasm
to create dist/lexer.js
.
On Windows it may be preferable to use the Linux subsystem.
After the Web Assembly build, the CJS build can be triggered via npm run build
.
Optimization passes are run with Binaryen prior to publish to reduce the Web Assembly footprint.
License
MIT