vscode-oniguruma

What is vscode-oniguruma?

The vscode-oniguruma package is a Node.js binding to the Oniguruma regular expressions library. It is primarily used in Visual Studio Code to power the textmate tokenizer, which is responsible for syntax highlighting based on TextMate grammars. The package allows for efficient regular expression matching and is optimized for use in text editors.

What are vscode-oniguruma's main functionalities?

Regular Expression Matching

This feature allows you to match strings against regular expressions. The code sample demonstrates how to load the Oniguruma WebAssembly binary, create a new OnigScanner with a regular expression, and find the next match in a given string starting from a specific position.

const { loadWASM, OnigScanner } = require('vscode-oniguruma');

// Load the WebAssembly binary for Oniguruma.
loadWASM(require('vscode-oniguruma/release/onig.wasm')).then(() => {
  const scanner = new OnigScanner(['\\w+']);
  const match = scanner.findNextMatchSync('hello world', 0);
  console.log(match);
});

Syntax Highlighting

This feature is used for syntax highlighting by matching keywords or patterns in a source code string. The code sample shows how to create an OnigScanner with a pattern for matching 'if' or 'else' keywords, create an OnigString from source code, and then find and log the matched keyword.

const { OnigScanner, OnigString } = require('vscode-oniguruma');

const scanner = new OnigScanner(['\\b(if|else)\\b']);
const source = new OnigString('if (condition) { return true; } else { return false; }');

const match = scanner.findNextMatchSync(source, 0);
if (match) {
  console.log(`Matched keyword: ${source.content.substring(match.captureIndices[0].start, match.captureIndices[0].end)}`);
}

Other packages similar to vscode-oniguruma

xregexp

XRegExp is an npm package that provides augmented, extensible regular expressions. It includes additional syntax and flags and offers features like named capture groups and format strings. While it does not use WebAssembly and is not specifically optimized for syntax highlighting in editors, it is a powerful tool for complex regular expression operations in JavaScript.

re2

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is built on the RE2 C++ library and prevents catastrophic backtracking by design. Unlike vscode-oniguruma, RE2 focuses on performance and safety in regular expression matching rather than syntax highlighting.

README

VS Code Oniguruma

Oniguruma bindings for VS Code. This library is used in VS Code and is not intended to grow to have general Oniguruma WASM bindings.

Installing

npm install vscode-oniguruma

API doc

See the main.d.ts file

Developing

• Clone the repository.
• Run git submodule init.
• Run git submodule update.
• Run git apply oniguruma.patch - This is necessary until https://github.com/kkos/oniguruma/issues/192 is fixed.
• Open the repository using the Remote - Containers extension, which will automatically create a docker container with the correct emscripten version and environment for building the WASM.
• Run npm install.
• Compile the oniguruma library with npm run build-onig (needed just once).
• Compile the .wasm with npm run build-wasm (needed every time the onig.cc file is changed).
• Compile the .js with npm run build-tsc or watch with npm run watch-tsc (needed every time the .ts files are changed).
• Package for releasing as UMD with npm run package

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.

License

MIT

This project incorporates material from other projects. Please see ThirdPartyNotices.txt

Thank you

Special thank you to @lieene for transfering the npm package name vscode-oniguruma to this project.