node-llama-cpp - npm Package Compare versions

Comparing version 2.1.0 to 2.1.1

llama/binariesGithubRelease.json

 {
-    "release": "b1107"
+    "release": "b1108"
 }

package.json

 {
   "name": "node-llama-cpp",
-  "version": "2.1.0",
+  "version": "2.1.1",
   "description": "node.js bindings for llama.cpp",
@@ -114,3 +114,2 @@ "main": "dist/index.js",
   "dependencies": {
-    "bytes": "^3.1.2",
     "chalk": "^5.3.0",
@@ -123,5 +122,3 @@ "cli-progress": "^3.12.0",
     "node-addon-api": "^7.0.0",
-    "node-downloader-helper": "^2.1.9",
     "node-gyp": "^9.4.0",
-    "node-stream-zip": "^1.15.0",
     "octokit": "^3.1.0",
@@ -128,0 +125,0 @@ "ora": "^7.0.1",

README.md

@@ -178,2 +178,19 @@ <div align="center">
 
+### Metal and CUDA support
+To load a version of `llama.cpp` that was compiled to use Metal or CUDA,
+you have to build it from source with the `--metal` or `--cuda` flag before running your code that imports `node-llama-cpp`.
+
+To do this, run this command inside of your project directory:
+```bash
+# For Metal support on macOS
+npx node-llama-cpp download --metal
+
+# For CUDA support
+npx node-llama-cpp download --cuda
+```
+
+> In order for `node-llama-cpp` to be able to build `llama.cpp` from source, make sure you have the required dependencies of `node-gyp` installed.
+> 
+> More info is available [here](https://github.com/nodejs/node-gyp#on-unix) (you don't have to install `node-gyp` itself, just the dependencies).
+
 ### CLI
@@ -210,2 +227,6 @@ ```
   -t, --nodeTarget       The Node.js version to compile llama.cpp for. Example: v18.0.0     [string]
+      --metal            Compile llama.cpp with Metal support. Can also be set via the NODE_LLAMA_CP
+                         P_METAL environment variable                     [boolean] [default: false]
+      --cuda             Compile llama.cpp with CUDA support. Can also be set via the NODE_LLAMA_CPP
+                         _CUDA environment variable                       [boolean] [default: false]
       --skipBuild, --sb  Skip building llama.cpp after downloading it     [boolean] [default: false]
@@ -225,2 +246,6 @@ -v, --version          Show version number                                               [boolean]
   -t, --nodeTarget  The Node.js version to compile llama.cpp for. Example: v18.0.0          [string]
+      --metal       Compile llama.cpp with Metal support. Can also be set via the NODE_LLAMA_CPP_MET
+                    AL environment variable                               [boolean] [default: false]
+      --cuda        Compile llama.cpp with CUDA support. Can also be set via the NODE_LLAMA_CPP_CUDA
+                     environment variable                                 [boolean] [default: false]
   -v, --version     Show version number                                                    [boolean]
@@ -227,0 +252,0 @@ ```

No alert changes

Improved metrics

Dependency count

13

-18.75%

Number of lines in readme file

314

8.65%

Worsened metrics

Total package byte prevSize

38785484

-0.44%

Dependency changes

• - Removed

  bytes@^3.1.2

• - Removed

  node-downloader-helper@^2.1.9

• - Removed

  node-stream-zip@^1.15.0

• - Removed

  bytes@3.1.2
  (transitive)

• - Removed

  node-downloader-helper@2.1.9
  (transitive)

• - Removed

  node-stream-zip@1.15.0
  (transitive)