New Case Study:See how Anthropic automated 95% of dependency reviews with Socket.Learn More
Socket
Sign inDemoInstall
Socket

llama-tokenizer-js

Package Overview
Dependencies
Maintainers
1
Versions
9
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

llama-tokenizer-js - npm Package Compare versions

Comparing version 1.2.0 to 1.2.1

3

package.json
{
"name": "llama-tokenizer-js",
"version": "1.2.0",
"version": "1.2.1",
"description": "JS tokenizer for LLaMA-based LLMs",
"main": "llama-tokenizer.js",
"types": "types.d.ts",
"scripts": {

@@ -7,0 +8,0 @@ "test": "node test-llama-tokenizer.js"

@@ -9,3 +9,3 @@ # 🦙 llama-tokenizer-js 🦙

Developed by [belladore.ai](https://belladore.ai)
Developed by [belladore.ai](https://belladore.ai) with contributions from [xenova](https://github.com/xenova), [blaze2004](https://github.com/blaze2004), [imoneoi](https://github.com/imoneoi) and [ConProgramming](https://github.com/ConProgramming).

@@ -114,2 +114,18 @@ ## Features

const tokenizer = new LlamaTokenizer(custom_vocab, custom_merge_data);
```
```
## Repo maintenance
Release steps:
1. node test-llama-tokenizer.js
2. open test.html
3. do you need to update this README?
4. bump version number in root package.json
5. push tokenizer changes to github
6. npm publish --dry-run
7. npm publish
8. bump version number in example-demo/package.json
9. cd example-demo && npm run build && live-server
10. push example demo changes to github
11. create release in github
SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc