Comparing version 0.0.8 to 0.0.9
{ | ||
"name": "cherow", | ||
"version": "0.0.8", | ||
"version": "0.0.9", | ||
"description": "", | ||
@@ -5,0 +5,0 @@ "main": "dist/cherow.js", |
198
README.md
# Cherow | ||
**WORK IN PROGRESS** | ||
Cherow is a very fast and unbeatable, standard-compliant ECMAScript parser written in ECMAScript. | ||
This is my playground code, and Cherow source code. It's still work in progress! | ||
It's close to 100% ECMAScript spec compatible. | ||
At this stage in development it's safe to use Cherow, and it covers the same thing as Esprima and Acorn | ||
does. It even parse a lot of cases the mentioned parsers fails on. | ||
**Note!** The source exist in the "dev" branch | ||
In fact. Cherow now parses everything after the ECMAScript specs. | ||
## Features | ||
However. There are a few cases that doesn't fail yet after the specs, but will be fixed now when I move on to fix and | ||
refactor some internals. | ||
- Full support for ECMAScript 2017 [(ECMA-262 8th Edition)](http://www.ecma-international.org/publications/standards/Ecma-262.htm) | ||
- 100 % TC262 Compatible | ||
- JSX | ||
- TypeScript (*in progress*) | ||
- Optional tracking of syntax node location (index-based and line-column) | ||
- 4300 unit tests with full code coverage | ||
**NOTE!!** No point to open issue tickets. I know what I'm doing : ) | ||
## ESNext features | ||
It's safe to use Cherow now, but I'm still not willing to move the code to Master branch until the internals are fixed, and | ||
the my remaining TODO are fixed. | ||
`Stage 3` features support. This need to be enabled with the `next` option | ||
Over 4300 unit tests should tell you that Cherow works! | ||
- Dynamic Import | ||
- Async generators | ||
- Async Await | ||
- Object spread | ||
- BigInt | ||
- Optional catch binding | ||
- Regular Expression's new `DotAll` flag | ||
Be aware that the code will change rapidly as I do progress. | ||
## Options | ||
## Current stage | ||
* `next` - Enables `ECMAScript Next` support and let you use proposals at `stage 3` or higher such as `Dynamic Import`. | ||
* `directives` - Enables support for [directive nodes](https://github.com/estree/estree/pull/152) | ||
* `raw` - Enables the raw property on literal nodes (*Esprima and Acorn feature*) | ||
* `comments` - Enables option to collect comments. Optional; Either array or function. Works like [Acorn](https://github.com/ternjs/acorn) onComment. | ||
* `tokens` - If enabled each found token will be returned as either an function or an array (*work in progres*) | ||
* `ranges` - Enables the start and characters offsets on the AST node. | ||
* `locations` - Enables location tracking. (*4 min fix, but on hold for now*) | ||
* `jsx` - Enables JSX | ||
* `ts` - Enables TypeScript (*coming very soon*) | ||
At current stage Cherow are working 100%, but not optimized. It's tested against both Acorn and Esprima and | ||
I'm also parsing huge libraries with this parser - both ES5 and ES6 code. | ||
To parse something, you can do: | ||
## Benchmarks | ||
```js | ||
implement { parseMOdule, parseScript } from 'cherow'; | ||
To be written | ||
// parse in sloppy mode | ||
parseScript('function foo() { return "bar"; } '); | ||
// parse inmodule code | ||
parseModule('function foo() { return "bar"; } '); | ||
## Contribution | ||
You are welcome to contribute. As a golden rule - always run benchmarks to verify that you haven't created any | ||
bottlenecks or did something that you shouldn't. | ||
// parse and get a ESTree output like Esprima | ||
*Terms of contribution:* | ||
parseScript('function foo() { return "bar"; } ', { raw: true, directives: true }); | ||
- Think twice before you try to implement anything | ||
- Minimum 1.5 mill ops / sec for light weight cases, and 800k - 1 mill ops / sec for "heavy" cases | ||
- Avoid duplicating the source code | ||
- Create tests that cover what you have implemented | ||
// parse and get a ESTree output like Acorn | ||
parseScript('function foo() { return "bar"; } ', { raw: true, ranges: true }); | ||
``` | ||
### Libraries fail to parse | ||
A few libraries fails to parse because I haven't implemented all code *yet*, or the code are unfinished. | ||
Example on this is that I haven't finished `for in` so one known issue is that it will throw. | ||
on `unexpected token in`. This is expected. **Note** This hasn't been completed yet because computed | ||
properties should be allowed with `in`. Something most open source parsers haven't implemented. | ||
And 2 lbraries can't be parsed due to issues in the source code in strict mode. This is out of scope for Cherow. | ||
I can also mention that Acorn have multiple regular expressions bugs in their source code. This has no | ||
effect just now because I haven't implemented my regular expression parser. But still worth to mention. | ||
*Here is the complete list I know of libraries that Cherow can't parse ATM* | ||
- MooTools 1.4.5 | ||
- jquery-1.9.1.js ( *fails on invalid in keyword* ) | ||
- yui-3.12.0 ( *fails on invalid in keyword* ) | ||
**Libraries with issues in source code** | ||
- DashDash (*only in strict or module code. Parses in sloppy mode*) | ||
- RX (*onaggregates module only*) | ||
## Roadmap | ||
Mostly get tests to fail where they should fail according to TC39. | ||
1. Finish expressions (*object and class clenup*) | ||
2. Finish statements (*slowly compare against TC39*) | ||
3. Finish module code | ||
4. Add back in TypeScript parser code (*it's written. Need to be added*) | ||
5. General cleanup and reduce amount of bitmasks | ||
6. Reduce code size | ||
## Important | ||
JSX are 99% completed and template are missing support for escaped sequences. This is done with purpose. TypeScript handle JSX and templates differently so this will be fixed after the TS parser code have been implemented. | ||
# Development process | ||
That is the most complex things I'm doing. I'm using `http://astexplorer.net/` to keep track on what fails and doesn't fail in various parsers. I'm also | ||
running `SpiderMonkey`, `V8`, and testing it against `NodejS` itself to validate if I'm doing the right things. | ||
Beside that. For every change I do in the code base, I'm parsing around 30 different libraries every time just to validate that I didn't break anything! | ||
On top of that I'm running various benchmarks. | ||
So everyone can make sure that the thing I'm pushing to this repo actually are working :) | ||
## Location tracker | ||
As it is now this parser have options you can use to get the same AST output as either Esprima or Acorn. By default Esprima doesnt have any location tracking on the node. Acorn has. For that use `ranges: true` as the option when you parse. | ||
Column and lines are in progress still. | ||
Note there exist a difference between how Esprima and Acorn calculating the ranges. You can see this in template string with an simple identifier. Acorn set the identifier start value to 2 (*after template head*). Esprima calculating it as 0. | ||
Due to this differences I'm still thinking how to do this. Example `Espree` uses Acorn, but uses Esprima location tracking layout. | ||
However. Column and lines can be activated with `ranges: true`, but will output wrong values due to the fact this is far from completed. | ||
I may end up adding a Esprima and Acorn mode option due to backward compability. | ||
## Error tracking | ||
This is an complex process. Mostly all open source parsers report either wrong location or wrong token position. One example here | ||
is Esprima wich is failing on invalid computed property. `({[x]})`. In this case Esprima will throw and report the last brace - `}` - as the wrong token. In fact this has nothing to do with the invalid computed shorthand property. | ||
Cherow are designed from ground up to fix all this things and report errors correctly. In mentioned case, Cherow will report the first bracket - `[` - as the wrong token. Wich is the start of the computed property. | ||
Here is an example on how I do it. | ||
It should fail on both `function static() { "use strict"; }` and `"use strict"; function static() {}` with correct error location. Note the `"use strict";` | ||
directive in the functions body. | ||
So the source code for it, will look like this: | ||
```js | ||
// if allready in strict mode code, thow | ||
this.error(Errors.UnexpectedStrictReserved); | ||
// ... else record current location and mark | ||
// that we found a reserved word | ||
this.errorLocation = this.trackErrorLocation(); | ||
this.flags |= Flags.ReservedWord; | ||
``` | ||
In the functions body I check if the parser state are in strict mode, and if that's the case, I check for the bitmask and throw the | ||
error at the recorded location. | ||
No magic! Just simple coding. | ||
## Performance | ||
Once again. Allmost all open source parsers have two or more deopts. Cherow are designed to avoid this. A good example is when you are accessing another object shape for checking values - `obj.type === "AssignmentExpression"`. This can *and* will cause an deopt. | ||
Performance is one out of many reasons why developing this parser take so long time. | ||
## TypeScript | ||
Cherow will be extended so it can parse TypeScript. This will add around 1450 extra lines of code to the code base. | ||
TypeScript parsing code will be added back in **after** Cherow are passing TC39 100%, and the code size have been optimized and reduced. | ||
Flow will not be supported. | ||
## TC39 | ||
At this stage 90% TC39 compatible. There exist a few tests that doesn't fail when they | ||
should. I'm working on it :) | ||
And there can be bugs in the bleeding edge cases like Stage 3 (*use next Option*). One example | ||
here is object rest spread. A few weeks ago the specs changed again, and I haven't updated it yet | ||
## Other | ||
### Tokenizing. | ||
It's there, but work in progress. Use `tokens: true`. This is work in progress. See *TODO*. | ||
### Comment collecting | ||
It's there, and working. It should work just like Acorn's `onComment`. Just do `comments: true` | ||
### Tolerate mode | ||
I still have to find a valid usecase for it. Once I do, I will add it. | ||
Sorry, the diff of this file is too big to display
Sorry, the diff of this file is not supported yet
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
178
487338
10988