Packages
16.59K
total weekly downloads
Current Co-maintainers
Former Co-maintainers
2 packages
tokenizers
Provides an implementation of today's most used tokenizers, with a focus on performances and versatility.
tokenizers-wasm
WebAssembly bindings to the Hugging Face tokenizers library, for fast tokenization in the browser.