batchwork

Batchwork

A unified batch API for AI providers. Submit thousands of LLM requests at roughly half the cost with a single call — batchwork handles JSONL, file uploads, inline submission, polling, and result parsing across every major provider.

📖 Full documentation: batchwork.dev

Install

npm install batchwork
# plus the provider package(s) you use:
npm install @ai-sdk/openai @ai-sdk/anthropic

batchwork depends only on ai. The @ai-sdk/* provider packages are optional peer dependencies — install only the ones you batch with. Requires Node.js 20 or newer.

Usage

import { batch } from "batchwork";
import { openai } from "@ai-sdk/openai";

const job = await batch({
  model: openai.chat("gpt-5.5"),
  requests: [
    { customId: "a", prompt: "Summarize: …" },
    { customId: "b", messages: [{ role: "user", content: "Translate: …" }] },
  ],
});

const results = await job.wait().then(() => job.collect());
for (const r of results) {
  console.log(r.customId, r.status, r.text);
}

Author requests in the same generateText shape you already use, pass the AI SDK models you already use, and get back one normalized result type correlated by customId.

Embeddings

Batch embeddings work the same way — pass a text embedding model and values, and get one vector per request back on result.embedding:

import { batchEmbeddings } from "batchwork";
import { openai } from "@ai-sdk/openai";

const job = await batchEmbeddings({
  model: openai.textEmbeddingModel("text-embedding-3-small"),
  requests: [
    { customId: "a", value: "The quick brown fox." },
    { customId: "b", value: "A lazy dog sleeps." },
  ],
});

const results = await job.wait().then(() => job.collect());
for (const r of results) {
  console.log(r.customId, r.embedding?.length);
}

Batch embeddings are available for OpenAI, Mistral, and Google Gemini — the providers whose batch API accepts embeddings. The rest throw a clear error: Anthropic, Groq, and xAI have no embedding model, and Together AI's batch API doesn't accept the embeddings endpoint.

Features

• One API, many providers — OpenAI, Anthropic, Google Gemini, Groq, Mistral, Together AI, and xAI.
• AI SDK native — author requests in the familiar generateText shape.
• Chat & embeddings — batch() for completions, batchEmbeddings() for vectors.
• ~50% cheaper — every request runs against the provider's batch window.
• Normalized results — unified status, text, usage, and error types regardless of provider.
• Server-ready — optional layers for managed polling, unified webhooks, and Next.js route handlers.
• Durable stores — drop-in Postgres (batchwork/postgres) and Upstash Redis (batchwork/redis) adapters for the poller, or bring your own.

Guides for models, the job handle, rehydration, the server layer, and Next.js handlers all live at batchwork.dev.

License

MIT © Hayden Bleasel