New Case Study:See how Anthropic automated 95% of dependency reviews with Socket.Learn More
Socket
Sign inDemoInstall
Socket

antlr4-c3

Package Overview
Dependencies
Maintainers
1
Versions
57
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

antlr4-c3 - npm Package Compare versions

Comparing version 2.2.3 to 3.0.0

coverage/ArrayType.ts.html

32

package.json
{
"name": "antlr4-c3",
"version": "2.2.3",
"version": "3.0.0",
"description": "A code completion core implementation for ANTLR4 based parsers",

@@ -18,7 +18,10 @@ "author": "Mike Lischke",

],
"main": "./out",
"main": "./lib/index",
"scripts": {
"prepublishOnly": "npm run test",
"test": "tsc --version && npm run generate && tsc && npm run eslint && mocha out/test",
"generate": "antlr4ts test/CPP14.g4 test/Expr.g4 test/Whitebox.g4 -no-listener -no-visitor -o test/generated -Xexact-output-dir",
"build": "tsc",
"prepublishOnly": "npm run generate && npm run test",
"test": "jest --testMatch [ \"**/tests/**/*.spec.ts\" ] --no-coverage --watchAll=false --max-worker=1",
"test-ci": "jest --testMatch [ \"**/tests/**/*.spec.ts\" ] --no-coverage --watchAll=false --silent",
"test-coverage": "jest --testMatch [ \"**/tests/**/*.spec.ts\" ] --coverage --silent",
"generate": "antlr4ts tests/CPP14.g4 tests/Expr.g4 tests/Whitebox.g4 -no-listener -no-visitor -o tests/generated -Xexact-output-dir",
"eslint": "eslint ."

@@ -30,17 +33,14 @@ },

"devDependencies": {
"@types/chai": "4.3.4",
"@types/mocha": "10.0.1",
"@types/node": "18.14.0",
"@typescript-eslint/eslint-plugin": "5.55.0",
"@typescript-eslint/parser": "5.55.0",
"@types/jest": "29.4.0",
"@types/node": "18.15.0",
"@types/unicode-properties": "1.3.0",
"@typescript-eslint/eslint-plugin": "5.54.1",
"@typescript-eslint/parser": "5.54.1",
"antlr4ts-cli": "0.5.0-alpha.4",
"chai": "4.3.7",
"eslint": "8.36.0",
"eslint-plugin-import": "2.27.5",
"eslint-plugin-jsdoc": "40.0.3",
"eslint-plugin-node": "11.1.0",
"eslint-plugin-jsdoc": "40.0.1",
"eslint-plugin-prefer-arrow": "1.2.3",
"eslint-plugin-promise": "6.1.1",
"mocha": "10.2.0",
"tslint": "6.1.3",
"jest": "29.5.0",
"ts-jest": "29.0.5",
"ts-node": "10.9.1",

@@ -47,0 +47,0 @@ "typescript": "5.0.2"

@@ -12,6 +12,4 @@ [![Build & Test](https://github.com/mike-lischke/antlr4-c3/actions/workflows/nodejs.yml/badge.svg?branch=master)](https://github.com/mike-lischke/antlr4-c3/actions/workflows/nodejs.yml)[![Downloads](https://img.shields.io/npm/dw/antlr4-c3?color=blue)](https://www.npmjs.com/package/antlr4-c3)

There have been quite a number of requests over the past years for getting support from ANTLR to create a code completion implementation, but so far that turned out as an isolated task with only custom solutions. This library aims to provide a common infrastructure for code completion implementations in a more general way, so that people can share their solutions and provide others with ideas to solve specific problems related to that.
This library provides a common infrastructure for code completion implementations. The c3 engine implementation is based on an idea presented a while ago under [Universal Code Completion using ANTLR3](https://soft-gems.net/universal-code-completion-using-antlr3/). There a grammar was loaded into a memory structure so that it can be walked through with the current input to find a specific location (usually the caret position) and then collect all possible tokens and special rules, which then describe the possible set of code completion candidates for that position. With ANTLR4 we no longer need to load a grammar, because the grammar structure is now available as part of a parser (via the ATN - [Augmented Transition Network](https://en.wikipedia.org/wiki/Augmented_transition_network)). The ANTLR4 runtime even provides the [LL1Analyzer](https://github.com/antlr/antlr4/blob/master/runtime/Java/src/org/antlr/v4/runtime/atn/LL1Analyzer.java) class, which helps with retrieving follow sets for a given ATN state, but has a few shortcomings and is in general not easy to use.
The c3 engine implementation is based on an idea presented a while ago under [Universal Code Completion using ANTLR3](https://soft-gems.net/universal-code-completion-using-antlr3/). There a grammar was loaded into a memory structure so that it can be walked through with the current input to find a specific location (usually the caret position) and then collect all possible tokens and special rules, which then describe the possible set of code completion candidates for that position. With ANTLR4 we no longer need to load a grammar, because the grammar structure is now available as part of a parser (via the ATN - [Augmented Transition Network](https://en.wikipedia.org/wiki/Augmented_transition_network)). The ANTLR4 runtime even provides the [LL1Analyzer](https://github.com/antlr/antlr4/blob/master/runtime/Java/src/org/antlr/v4/runtime/atn/LL1Analyzer.java) class, which helps with retrieving follow sets for a given ATN state, but has a few shortcomings and is in general not easy to use.
With the Code Completion Core implementation things become a lot easier. In the simplest setup you only give it a parser instance and a caret position and it will return the candidates for it. Still, a full code completion implementation requires some support code that we need to discuss first before we can come to the actual usage of the c3 engine.

@@ -86,2 +84,3 @@

# Getting Started
With this knowledge we can now look at a simple code example that shows how to use the engine. For further details check the unit tests for this node module (under the test/ folder).

@@ -175,3 +174,5 @@

# Fine Tuning
## Ignored Tokens
As mentioned above in the base setup the engine will only return lexer tokens. This will include your keywords, but also many other tokens like operators, which you usually don't want in your completion list. In order to ease usage you can tell the engine which lexer tokens you are not interested in and which therefor should not appear in the result. This can easily be done by assigning a list of token ids to the `ignoredTokens` field before you invoke `collectCandidates()`:

@@ -190,5 +191,7 @@

## Preferred Rules
As mentioned already the `preferredRules` field is an essential part for getting more than just keywords. It lets you specify the parser rules that are interesting for you and should include the rule indexes for the entities we talked about in the code completion breakdown paragraph above. Whenever the c3 engine hits a lexer token when collecting candidates from a specific ATN state it will check the call stack for it and, if that contains any of the preferred rules, will select that instead of the lexer token. This transformation ensures that the engine returns contextual information which can actually be used to look up symbols.
## Constraining the Search Space
Walking the ATN can at times be quite expensive, especially for complex grammars with many rules and perhaps (left) recursive expression rules. I have seen millions of visited ATN states for complex input, which will take very long to finish. In such cases it pays off to limit the engine to just a specific rule (and those called by it). For that there is an optional parser rule context parameter in the `collectCandidates()` method. If a context is given the engine will never look outside of this rule. It is necessary that the specified caret position lies within that rule (or any of those called by it) to properly finish the ATN walk.

@@ -201,2 +204,3 @@

## Selecting the Right Caret Position
It might sound weird to talk about such a trivial thing like the caret position but there's one thing to consider, which makes this something you have to think about. The issue is the pure token index returned by the token stream and the visual appearance on screen. This image shows a typical scenario:

@@ -211,2 +215,3 @@

# Debugging
Sometimes you are not getting what you actually expect and you need take a closer look at what the c3 engine is doing. For this situation a few fields have been added which control some debug output dumped to the console:

@@ -223,2 +228,12 @@

### 3.0.0
BREAKING CHANGES: With this major version release the API has been changed to make it more consistent and easier to use. The most important changes are:
- All the classes in the SymbolTable.ts file have been split into separate files.
- The main Symbol class has been renamed to `BaseSymbol` to avoid confusion and trouble with the Javascript `Symbol` class.
- The package works now with Typescript 5.0 and above.
- The tests have been organized into a separate sub project, which is no longer built with the main project. Instead tests files are transpiled on-the-fly (using `ts-jest`) when running the tests. These transpiled files are never written to disk.
- Symbol creation functions (like `SymbolTable.addNewSymbolOfType`) now allow Typescript to check the given parameters for the class type. You will now have to provide the correct parameter list for the symbol type you want to create. This is a breaking change, because the old version allowed you to pass any parameter list to any symbol creation function.
### 2.2.3

@@ -225,0 +240,0 @@

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc