Socket
Socket
Sign inDemoInstall

js-tokens

Package Overview
Dependencies
0
Maintainers
1
Versions
23
Alerts
File Explorer

Advanced tools

Install Socket

Protect your apps from supply chain attacks

Install

js-tokens

Tiny JavaScript tokenizer.

    9.0.0latest
    GitHub
    npm

Version published
Maintainers
1
Weekly downloads
37,882,720
decreased by-13.77%

Weekly downloads

Package description

What is js-tokens?

The js-tokens npm package is a lightweight, regex-based lexer for JavaScript tokens. It can tokenize JavaScript code without the overhead of a full parser, making it useful for tasks like syntax highlighting or extracting specific tokens from code.

What are js-tokens's main functionalities?

Tokenizing JavaScript code

This feature allows you to tokenize a string of JavaScript code into an array of token objects. Each token object describes a syntactic element of the code, such as a keyword, identifier, number, string, or operator.

const jsTokens = require('js-tokens');
const tokens = [...jsTokens('var x = 42;')];
console.log(tokens);

Other packages similar to js-tokens

Changelog

Source

Version 9.0.0 (2024-02-08)

  • Added: Support for ES2023: #! hashbang comments at the start of files.

Readme

Source

js-tokens

The tiny, regex powered, lenient, almost spec-compliant JavaScript tokenizer that never fails.

const jsTokens = require("js-tokens");

const jsString = 'JSON.stringify({k:3.14**2}, null /*replacer*/, "\\t")';

Array.from(jsTokens(jsString), (token) => token.value).join("|");
// JSON|.|stringify|(|{|k|:|3.14|**|2|}|,| |null| |/*replacer*/|,| |"\t"|)

➡️ Full readme

Keywords

FAQs

Last updated on 08 Feb 2024

Did you know?

Socket installs a GitHub app to automatically flag issues on every pull request and report the health of your dependencies. Find out what is inside your node modules and prevent malicious activity before you update the dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc