Text Processing

Virtual file format for text processing

Determine the East Asian Width of a Unicode character

A convertor between XML text and Javascript object / JSON text.

Bindings for RE2: fast, safe alternative to backtracking regular expression engines.

Markdown parser, done right. 100% Commonmark support, extensions, syntax plugins, high speed - all in one.

easier than regex string matching patterns for urls and other strings. turn strings into data or data into strings.

retext plugin to serialize prose

vfile utility to read and write to the file system

Elegant console output, borrowed from Yarn

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

Javascript markdown parsing, made simple

A sensible Markdown parser for javascript

A module for node.js and the browser that takes in text and returns text that is stripped of stopwords. Has pre-defined stopword lists for 62 languages and also takes lists with custom stopwords as input.

Webpack plugin to use in addition to [extract-text-webpack-plugin](https://github.com/webpack/extract-text-webpack-plugin) to create a second css bundle, processed to be rtl.

Teams Toolkit CLI is a text-based command line interface that can help scaffold, validate, and deploy applications for Microsoft Teams from the terminal or a CI/CD process.

novel-segment segment data

Helps to prevent widow words in a text

DevExpress Rich Text Editor is an advanced word-processing tool designed for working with rich text documents.

TeamsFx CLI a text-based command line interface that can help scaffold, validate, and deploy applications for Microsoft Teams from the terminal or a CI/CD process.


Count the number of OpenAI tokens in a string. Supports all OpenAI Text models (text-davinci-003, gpt-3.5-turbo, gpt-4)

Anonymize-NLP is a lightweight and robust package for text anonymization. It uses Natural Language Processing (NLP) and Regular Expressions (Regex) to identify and mask sensitive information in a string.,

A convertor between XML text and Javascript object / JSON text. Forked to add Graphite specific features.

Basic library to roughly determine the language of input text

Plugin for Remarkable to process embedded math expressions in Markdown text.

Util collection for Japanese text processing. Hiraganize, Katakanize, and Romanize.

Chinese word segmentation 簡繁中文分词模块 以網路小說為樣本

🔪 chunk/split a string by length without cutting/truncating words.

Javascript SDK for Sensible, the developer-first platform for extracting structured data from documents so that you can build document-automation features into your SaaS products

原版 node-segment 的格式

Node PDF is a set of tools that takes in PDF files and converts them to usable formats for data processing. The library supports both extracting text from searchable pdf files as well as performing OCR on pdfs which are just scanned images of text

A converter between XML text and Javascript object / JSON text.

Naive Bayes Text Classifier

Configurable BM25 Text Search Engine with simple semantic search support

Util collection for Japanese text processing. Hiraganize, Katakanize, and Romanize.

Javascript BPE Encoder Decoder for GPT-2 / GPT-3. The "gpt-3-encoder" module provides functions for encoding and decoding text using the Byte Pair Encoding (BPE) algorithm. It can be used to process text data for input into machine learning models, or to

Multi Languages Detection for Text-Mining and Natural Language Processing - True ITK - Open Source

A highly efficient, isomorphic, full-featured, multilingual text search engine library, providing full-text search, fuzzy matching, phonetic scoring, document indexing and more, with micro JSON state hydration/dehydration in-browser and server-side.

Node and Browser env supported WebAssembly version of fastText: Library for efficient text classification and representation learning.

Make your app understand language. Summarize conversations, categorize articles, and more.

Convert YAML into JSON. Supports YAML frontmatter, YAML multidocs and other mixed text-and-yaml documents. Can process Textile, Markdown and AsciiDoc strings. Works on the command-line and in node.js.