Run ONNX and TensorFlow inference in the browser. A thin wrapper on top of tract.
The Open Neural Network Exchange is a format which many popular libraries like PyTorch, TensorFlow and MXNet can export to which allows tractjs to run neural networks from (almost) any library.
There is currently one other usable ONNX runner for the browser, ONNX.js. There are a couple of things tractjs does better:
tractjs.min.js which contains the inlined WASM and WebWorker. The WASM backend of ONNX.js can not as easily be used without a build system.There are however also some downsides to tractjs. See the FAQ.
<html>
<head>
<meta charset="utf-8" />
<script src="https://unpkg.com/tractjs/dist/tractjs.min.js"></script>
<script>
const model = new tractjs.Model("path/to/your/model");
model
.predict([new tractjs.Tensor(new Float32Array([1, 2, 3, 4]), [2, 2])])
.then((preds) => {
console.log(preds);
});
</script>
</head>
</html>
npm install tractjs
import * as tractjs from "tractjs";
const model = new tractjs.Model("path/to/your/model");
model
.predict([new tractjs.Tensor(new Float32Array([1, 2, 3, 4]), [2, 2])])
.then((preds) => {
console.log(preds);
});
tractjs now runs in Node.js! Models are fetched from the file system.
const tractjs = require("tractjs");
const model = new tractjs.Model("./path/to/your/model");
model
.predict([new tractjs.Tensor(new Float32Array([1, 2, 3, 4]), [2, 2])])
.then((preds) => {
console.log(preds);
});
My model with dynamic input dimensions doesn't work. Why?
Currently, tract requires fully determined input dimensions to optimize a model. There are two options:
optimize off:const model = new tractjs.Model("path/to/your/model", {
optimize: false,
});
This will however significantly impact performance.
const model = new tractjs.Model("path/to/your/model", {
inputFacts: {
0: ["float32", [1, 3, 224, 224]],
},
});
Be aware that the model will only work properly with inputs of this exact shape though.
There is ongoing work in tract to allow dynamically sized inputs.
What about size?
At the time of writing, tractjs is very large for web standards (8.5MB raw, 2.5MB gzipped). This is due to tract being quite large, and due to some overhead from inlining the WASM. But it's not as bad as it sounds. You can load tractjs lazily along your demo, where you will likely have to load significantly large weights too.
If you are working on a very size-sensitive application, get in touch and we can work on decreasing the size. There are some more optimizations to be done (e. g. an option not to inline WASM, and removing panics from the build). There is also ongoing work in tract to decrease size.
What about WebGL / WebNN support?
tractjs are bindings to the tract Rust library which was originally not intended to be run on the web. WebGL / WebNN support would be great, but would require lots of web-specific changes in tract so it is currently not under consideration.
All original work licensed under either of
Contributions are very welcome! See CONTRIBUTING.md.
FAQs
A library for running ONNX and TensorFlow inference in the browser.
The npm package tractjs receives a total of 509 weekly downloads. As such, tractjs popularity was classified as not popular.
We found that tractjs demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.
Security News
PyPI now supports digital attestations, enhancing security and trust by allowing package maintainers to verify the authenticity of Python packages.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.