llama3-tokenizer-js - npm Package Compare versions

Comparing version 1.1.4 to 1.1.5

package.json

 {
   "name": "llama3-tokenizer-js",
-  "version": "1.1.4",
+  "version": "1.1.5",
   "description": "JS tokenizer for LLaMA 3",
@@ -5,0 +5,0 @@ "main": "bundle/llama3-tokenizer-with-baked-data.js",

src/create-bundle.js

@@ -57,2 +57,11 @@ import fs from 'node:fs';
             });
+
+            // Someone on GitHub requested that for them to load the file it has to end in .cjs,
+            // and I didn't want to break existing filepaths, so we'll just have the same bundle in 2 different files.
+            fs.writeFile('../bundle/commonjs-llama3-tokenizer-with-baked-data.cjs', content, err4 => {
+                if (err4) {
+                  console.error(err4)
+                  return
+                }
+            });
         })
@@ -59,0 +68,0 @@

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

12882728

increased by33.28%

Number of package files

15

increased by7.14%

Lines of code

46086

increased by39.54%

No dependency changes