batchwork

Batchwork

A unified batch API for AI providers. Submit thousands of LLM requests at roughly half the cost with a single call — batchwork handles JSONL, file uploads, inline submission, polling, and result parsing across every major provider.

📖 Full documentation: batchwork.dev

Install

npm install batchwork
# plus the provider package(s) you use:
npm install @ai-sdk/openai @ai-sdk/anthropic

batchwork depends only on ai. The @ai-sdk/* provider packages are optional peer dependencies — install only the ones you batch with. Requires Node.js 20 or newer.

Usage

import { batch } from "batchwork";
import { openai } from "@ai-sdk/openai";

const job = await batch({
  model: openai.chat("gpt-5.5"),
  requests: [
    { customId: "a", prompt: "Summarize: …" },
    { customId: "b", messages: [{ role: "user", content: "Translate: …" }] },
  ],
});

const results = await job.wait().then(() => job.collect());
for (const r of results) {
  console.log(r.customId, r.status, r.text);
}

Author requests in the same generateText shape you already use, pass the AI SDK models you already use, and get back one normalized result type correlated by customId.

Embeddings

Batch embeddings work the same way — pass a text embedding model and values, and get one vector per request back on result.embedding:

import { batch } from "batchwork";
import { openai } from "@ai-sdk/openai";

const job = await batch.embeddings({
  model: openai.embeddingModel("text-embedding-3-small"),
  requests: [
    { customId: "a", value: "The quick brown fox." },
    { customId: "b", value: "A lazy dog sleeps." },
  ],
});

const results = await job.wait().then(() => job.collect());
for (const r of results) {
  console.log(r.customId, r.embedding?.length);
}

Batch embeddings are available for OpenAI, Mistral, and Google Gemini — the providers whose batch API accepts embeddings. The rest throw a clear error: Anthropic, Groq, and xAI have no embedding model, and Together AI's batch API doesn't accept the embeddings endpoint.

Images

Generate images in bulk — pass an image model and prompts, and get base64 images back on result.images:

import { batch } from "batchwork";
import { openai } from "@ai-sdk/openai";

const job = await batch.images({
  model: openai.image("gpt-image-2"),
  requests: [
    { customId: "a", prompt: "A red bicycle against a brick wall." },
    { customId: "b", prompt: "A watercolor painting of a sleeping cat." },
  ],
});

const results = await job.wait().then(() => job.collect());
for (const r of results) {
  for (const image of r.images ?? []) {
    // Inline base64 (`image.data` + `image.mediaType`), or a hosted
    // `image.url` for providers that return one (e.g. xAI batch).
    console.log(r.customId, image.mediaType ?? image.url);
  }
}

Batch image generation is available for OpenAI (/v1/images/generations, e.g. gpt-image-2), Google Gemini image models (e.g. gemini-2.5-flash-image), and xAI (/v1/images/generations, e.g. grok-imagine-image-quality); other providers throw a clear error. Generation only — image editing, video, and Google's Imagen models aren't batch-supported, and Together AI's batch API is chat/audio only. OpenAI and Google return inline base64 on image.data; xAI batch returns signed image.urls that expire ~1h after completion, so download them promptly.

Features

• One API, many providers — OpenAI, Anthropic, Google Gemini, Groq, Mistral, Together AI, and xAI.
• AI SDK native — author requests in the familiar generateText shape.
• Chat, embeddings & images — batch() for completions, batch.embeddings() for vectors, batch.images() for image generation.
• ~50% cheaper — every request runs against the provider's batch window.
• Normalized results — unified status, text, usage, and error types regardless of provider.
• Server-ready — optional layers for managed polling, unified webhooks, and Next.js route handlers.
• Durable stores — drop-in Postgres (batchwork/postgres) and Upstash Redis (batchwork/redis) adapters for the poller, or bring your own.

Guides for models, the job handle, rehydration, the server layer, and Next.js handlers all live at batchwork.dev.

License

MIT © Hayden Bleasel