@qvac/sdk

QVAC is an open-source, cross-platform system for building local-first, peer-to-peer AI applications. QVAC runs on Bare by Holepunch, a lightweight, cross-platform JavaScript runtime.

QVAC SDK

QVAC SDK is the canonical entry point to QVAC. Written in TypeScript, it provides all QVAC capabilities through a unified interface while also abstracting away the complexity of running your application in a JS environment other than Bare.

For the comprehensive reference, see official QVAC documentation.

Requirements

Supported JS environments: Bare, Node.js, Expo and Bun.

Installation

Steps

• Authenticate to Tether’s GitHub packages
• Install package and dependencies

Step 1: authenticate to Tether’s GitHub packages

• Create a GitHub personal access token (classic) with scope read:packages.

• Add the token to your environment variables. For example, on macOS/Linux:

export NPM_TOKEN=<token_generated_in_previous_step>

• In the root of your project, create file .npmrc with the following content:

@qvac:registry=https://registry.npmjs.org
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

Alternatively, you can create the .npmrc file in your home directory, for user-level (global) configuration.

• Confirm authentication to Tether’s GitHub packages:

npm whoami --registry=https://npm.pkg.github.com

It should return your GitHub username.

Step 2: Install package and dependencies

• Install in your project using the package manager of your choice:

npm i @qvac/sdk

[!TIP]
If you can't get the package via Tether's private GitHub package registry, see section Build from source.

On Linux

[!IMPORTANT]
If you're on Linux ensure that the Vulkan SDK is installed (e.g. apt install vulkan-sdk on Ubuntu)

On Expo

• Install the peer dependencies for Expo/React Native support:

npm i expo-file-system react-native-bare-kit

• If you are on Android, bump your minSdkVersion to 29. You can do so by adding ext { minSdkVersion=29 } to your android/build.gradle or by using expo-build-properties.

• Add the QVAC Expo plugin to your app.json:

export default {
  expo: {
    plugins: ["@qvac/sdk/expo-plugin"],
  },
};

• Prebuild your project to generate the native files:

npx expo prebuild

• Build and run it on a physical device:

npx expo run:ios --device
# or
npx expo run:android --device

[!IMPORTANT]
Due to limitations with llamacpp, QVAC currently does not run on emulators. You must use a physical device.

Quickstart

You can run the self-contained example below:

import { loadModel, completion, unloadModel } from "@qvac/sdk";

// Load model into memory
const modelId = await loadModel(
  "pear://afa79ee07c0a138bb9f11bfaee771fb1bdfca8c82d961cff0474e49827bd1de3/Llama-3.2-1B-Instruct-Q4_0.gguf",
  {
    modelType: "llm",
  },
);

// Use the loaded model
const response = completion({
  modelId,
  history: [{ role: "user", content: "What is the capital of France?" }],
  stream: false,
});
const text = await response.text;
console.log(text);

// Enable KV cache for faster subsequent completions
const cachedResponse = completion({
  modelId,
  history: [
    { role: "user", content: "What is the capital of France?" },
    { role: "assistant", content: "The capital of France is Paris." },
    { role: "user", content: "What about Germany?" },
  ],
  stream: true,
  kvCache: true, // enables caching for faster inference
});

// You can use the loaded model multiple times

// Unload model to free up system resources
await unloadModel({ modelId });
process.exit();

For more on how to use QVAC SDK, see SDK at QVAC documentation.

Build from source

Use the Bun package manager:

bun i

bun run build  # or `watch` for hotreload

bun run build:pack

This outputs a tarball under dist/qvac-sdk-{version}.tgz that you can install in your project, e.g.:

npm i path/to/qvac-sdk-0.3.0.tgz

Examples

In the ./examples subdirectory, you will find scripts demonstrating SDK usage. To try any of them:

• Follow the steps in section Build from source.
• Run using Bare, Node.js, or Bun as the runtime:

# With Bare
bun run bare:example dist/examples/path/to/example.js

# With Node
node dist/examples/path/to/example.js

# With bun, straight from source
bun run examples/path/to/example.ts

Logging

QVAC SDK includes built-in log propagation that streams model logs from the worker process to your client application in real-time. This gives you visibility into what's happening inside your models during loading, inference, and other operations.

Usage

Simply pass a logger when loading your model:

import { getLogger, loadModel, completion, unloadModel } from "@qvac/sdk";

// Create a logger for your application
const logger = getLogger("my-app", {
  level: "debug", // Set log level
  transports: [
    // Optional: add custom log handlers
    (level, namespace, message) => {
      console.log(`[${level.toUpperCase()}] ${namespace}: ${message}`);
    },
  ],
});

// Load model with logging - RPC stream starts automatically
const modelId = await loadModel(
  "pear://afa79ee07c0a138bb9f11bfaee771fb1bdfca8c82d961cff0474e49827bd1de3/Llama-3.2-1B-Instruct-Q4_0.gguf",
  {
    modelType: "llm",
    logger, // Enable log streaming
  },
);

// Now you'll see model logs in your console as operations happen
for await (const token of completion({
  modelId,
  history: [{ role: "user", content: "Hello!" }],
}).tokenStream) {
  process.stdout.write(token);
}

await unloadModel({ modelId }); // Logging stream closes automatically

What You'll See

When logging is enabled, you'll see real-time logs from the underlying model libraries:

[DEBUG] llamacpp:llm: Loading model weights...
[INFO] llamacpp:llm: Model loaded successfully, vocab_size=32000
[DEBUG] llamacpp:llm: Starting inference...
[DEBUG] llamacpp:llm: Inference completed, tokens=12

This works for all model types (LLM, Whisper, NMT, Embeddings) and provides valuable insight into model performance and behavior.

Completion

• llama.cpp with local files: examples/llamacpp-filesystem.ts
• llama.cpp with Hyperdrive: examples/llamacpp-hyperdrive.ts
• llama.cpp with HTTP: examples/llamacpp-http.ts
• llama.cpp with file logging: examples/llamacpp-file-logging.ts
• llama.cpp with tools/function calls: examples/llamacpp-tools.ts
• llama.cpp with tools and routing: examples/llamacpp-tools-routing.ts
• llama.cpp with multimodal inference: examples/llamacpp-multimodal.ts
• llama.cpp with KV cache: examples/kv-cache-example.ts

Transcription

• whisper.cpp transcription: examples/whispercpp-transcription.ts
• Microphone recording: examples/microphone-record-transcription.ts

Embeddings

• Embedding with Hyperdrive: examples/embed-hyperdrive.ts
• RAG example with HyperDB: examples/rag-hyperdb.ts
• RAG with workspaces: examples/rag-hyperdb-workspaces.ts (Demonstrates workspace isolation and chunking)
• RAG example with LanceDB (desktop only): examples/rag-lancedb.ts
• RAG example with ChromaDB (desktop only): examples/rag-chromadb.ts (Note: Requires ChromaDB server running)
• RAG example with SQLite-Vector (desktop only): examples/rag-sqlite.ts (Note: Uses SQLite-Vector WASM)

Translation

• Marian OPUS translation: examples/translation-opus.ts
• Indic language translation: examples/translation-indic.ts
• LLM-based translation: examples/translation-llm.ts

Text to Speech

• Text to Speech: examples/text-to-speech.ts

Note: To run the text-to-speech example, espeak data is required which can be obtained from here.

Multimodel

• Load multiple models simultaneously: examples/multi-model-demo.ts

Delegated inference

• Provider example: examples/delegated-inference/provider.ts
• Consumer example: examples/delegated-inference/consumer.ts

Note: Set the QVAC_HYPERSWARM_SEED env var to ensure that the provider's uses the same keypair (i.e. the public key doesn't change on every run). Note: ⚠️ Consumer does not handle reconnection yet.

Download Lifecycle

• Pause and resume download: examples/download-with-cancel.ts

Sharded Models

• Load sharded models from Hyperdrive: examples/llamacpp-sharded.ts
• Load models from local filesystem: examples/llamacpp-filesystem.ts

Note: Sharded models automatically download all parts sequentially with detailed progress tracking per shard. For local sharded models, pass the path to the first shard file (e.g., model-00001-of-00005.gguf), and remaining shards will be loaded automatically.

Blind Relays

Blind relays help establish peer connections through NAT/firewalls by routing traffic through relay nodes.

• Model downloads via Hyperdrive: examples/download-with-blind-relays.ts
• Delegated inference: You can reuse the same pattern for delegated inference by calling setConfig({ swarmRelays: [...relayPublicKeys] }) before starting your provider/consumer.

Note: The example uses mock relay keys. In real deployments, you must use your own relay servers or trusted public relays.

Known issues

• One off scripts don't terminate / hang

  • Add a process.exit() call at the end, we're working on it.

• I'm building my Expo app on iOS but I don't see logs from QVAC

  • npx expo start
  • Open the ios folder in Xcode
  • Start the app with the play ▶️ button
  • View -> Debug area -> Activate console
  • You should see the system logs on the right pane
  • ⚠️ This only works when starting the app from Xcode