onnxruntime-node

ONNX Runtime Node.js Binding

ONNX Runtime Node.js binding enables Node.js applications to run ONNX model inference.

Usage

Install the latest stable version:

npm install onnxruntime-node

Install the nightly version:

npm install onnxruntime-node@dev

Refer to ONNX Runtime JavaScript examples for samples and tutorials.

Requirements

ONNXRuntime works on Node.js v16.x+ (recommend v20.x+) or Electron v15.x+ (recommend v28.x+).

The following table lists the supported versions of ONNX Runtime Node.js binding provided with pre-built binaries.

EPs/Platforms	Windows x64	Windows arm64	Linux x64	Linux arm64	MacOS x64	MacOS arm64
CPU	✔️	✔️	✔️	✔️	✔️	✔️
WebGPU	✔️ [1]	✔️ [1]	✔️ [1]	❌ [2]	✔️ [1]	✔️ [1]
DirectML	✔️	✔️	❌	❌	❌	❌
CUDA	❌	❌	✔️[3]	❌	❌	❌
CoreML	❌	❌	❌	❌	✔️	✔️

• [1]: WebGPU support is currently experimental.
• [2]: WebGPU support is not available on Linux arm64 yet in the pre-built binaries.
• [3]: CUDA v12. See CUDA EP Installation for details.

To use on platforms without pre-built binaries, you can build Node.js binding from source and consume it by npm install <onnxruntime_repo_root>/js/node/. See also instructions for building ONNX Runtime Node.js binding locally.

GPU Support

Right now, the Windows version supports WebGPU execution provider and DML execution provider. Linux x64 can use CUDA and TensorRT.

CUDA EP Installation

To use CUDA EP, you need to install the CUDA EP binaries. By default, the CUDA EP binaries are installed automatically when you install the package. If you want to skip the installation, you can pass the --onnxruntime-node-install=skip flag to the installation command.

npm install onnxruntime-node --onnxruntime-node-install=skip

You can also use this flag to specify the version of the CUDA: (v11 or v12) CUDA v11 is no longer supported since v1.22.

License

License information can be found here.

Similar packages

Other packages similar to onnxruntime-node

tensorflow

TensorFlow is an open-source platform for machine learning. It provides a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML, and developers easily build and deploy ML-powered applications. Unlike onnxruntime-node, TensorFlow is not limited to ONNX models and offers a broader range of functionalities including model training.

ml5

ml5.js is a friendly high-level interface to TensorFlow.js, designed to make machine learning accessible to a broad audience. It provides pre-trained models and simplifies the process of integrating machine learning into web applications. Compared to onnxruntime-node, ml5.js is more focused on ease of use and accessibility, particularly for beginners and educators.