@runinfra/sdk

RunInfra TypeScript SDK

Access optimized RunInfra deployments through the verified public gateway.

Requires Node.js 18 or newer.

Install

npm install @runinfra/sdk

Modality status (v0.2.0)

This SDK is in beta. The surfaces below have different verification levels:

Modality	Surface	Status
LLM	client.chat.completions.create, client.responses.create	Beta. Typed helpers for verified LLM and vision-language deployments.
Embeddings	client.embeddings.create	Beta. Typed helper for verified embedding deployments.
Images	client.images.generate	Preview. Available when the deployment exposes image generation.
Audio (TTS)	client.audio.speech.create	Preview. Available when the deployment exposes speech generation.
Audio (ASR)	client.audio.transcriptions.create	Preview. Available when the deployment exposes transcription.
Webhooks	client.webhooks.verifySignature, client.webhooks.constructEvent, verifyWebhookSignature, constructWebhookEvent	Local verification helpers only; delivery management is outside the public SDK surface
Voice pipeline	client.voice.pipeline.create	Preview. Pipeline-scoped helper for co-located audio-to-response deployments.

The dashboard only shows snippets for operations the selected deployment supports. If a route is unsupported for a deployment, the SDK returns a typed error instead of silently falling back to another operation.

Create a client

Use a workspace-scoped key to reach verified active deployments through the model field. In the RunInfra dashboard, open Settings, API Keys, Create key, and keep Scope set to Workspace.

The Deploy tab can create a pipeline-scoped key for one optimized pipeline. The one-time secret is shown once after creation. Store it as RUNINFRA_API_KEY for app snippets before leaving the page.

After an optimization run finishes, open the Deploy view from the dashboard. Deploy only shows SDK operations that the verified endpoint supports, so copy the native or OpenAI-compatible snippet from there instead of guessing a route.

import { RunInfra } from "@runinfra/sdk";

const apiKey = process.env.RUNINFRA_API_KEY;
if (!apiKey) throw new Error("Set RUNINFRA_API_KEY before running this snippet.");

const client = new RunInfra({
  apiKey,
});

Use pipelineId when the key or integration should be locked to one optimized pipeline.

const apiKey = process.env.RUNINFRA_API_KEY;
if (!apiKey) throw new Error("Set RUNINFRA_API_KEY before running this snippet.");

const client = new RunInfra({
  apiKey,
  pipelineId: "pipe_123",
});

The default base URL is https://api.runinfra.ai/v1. pipelineId is trimmed and URL-encoded before it is added to the base URL. Use either pipelineId with the default base URL, or a pipeline-scoped baseURL such as https://api.runinfra.ai/v1/pipe_123. If both point to the same pipeline, the SDK keeps the URL scoped once. RunInfra generated native SDK snippets prefer pipelineId with the root https://api.runinfra.ai/v1 base URL. OpenAI-compatible snippets use the pipeline-scoped base URL because the OpenAI SDK has no RunInfra pipeline option. Custom base URLs must use http or https. Other schemes and malformed URLs are rejected before a bearer API key can be sent. Remote custom base URLs must use https. Plain http is accepted only for local development hosts: localhost, 127.0.0.1, 0.0.0.0, and [::1]. Custom base URLs must not include usernames or passwords. Custom base URLs must not include query strings or fragments.

Server-side only

The TypeScript SDK targets Node.js 18 or newer. RunInfra API keys are bearer secrets. Do not put RUNINFRA_API_KEY in browser code and do not initialize this SDK in public client bundles with a secret API key. The SDK fails closed when it detects a browser runtime; keep calls on a Node.js server route, backend proxy, API service, or backend job. Browser apps should call your own server first, then your server calls RunInfra with the workspace or pipeline-scoped key. Direct browser token flows are not supported by the public SDK. If you are deliberately using a controlled non-public browser-like runtime, pass dangerouslyAllowBrowser: true and own that risk.

Unknown TypeScript client option keys are rejected so typos such as baseUrl or api_key do not silently change the gateway, authentication, timeout, retry, or runtime-safety behavior. Use baseURL for custom server-side gateway URLs.

Responses and streaming

const stream = await client.responses.create({
  model: "llama-3.1-8b",
  input: "Hello",
  max_output_tokens: 512,
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta ?? "");
  }
}

console.log(stream.requestId);

Breaking out of the for await loop cancels the underlying SSE reader and releases the reader lock. If you manually advance the stream iterator, call return() on that iterator when you stop early so local response resources are released. Streaming transport-level backend cancellation is best effort, and streaming calls are not automatically retried.

RunInfra /v1/responses is a chat-completions compatibility adapter. The gateway converts supported input and instructions values into chat messages, forwards the supported request through the chat-completions serving path, and rewraps the result into a Responses-style envelope. It does not claim full OpenAI Responses state, include, reasoning, tool, conversation-item, or background-job semantics.

Supported public routes

• models.list()
• models.retrieve(model)
• responses.create()
• chat.completions.create()
• embeddings.create()
• audio.speech.create()
• audio.transcriptions.create()
• images.generate()
• voice.pipeline.create()

OpenAI-compatible parameter scope

The native SDK validates the minimum request fields locally, then forwards OpenAI-style JSON or multipart fields that preserve the typed response shape. The typed native SDK subset is:

• Chat Completions: model, messages, stream, temperature, top_p, max_tokens, stop, presence_penalty, frequency_penalty, user, and metadata; streaming usage chunks are covered separately with stream_options.include_usage.
• Responses: model, input, stream, instructions, temperature, top_p, tools, tool_choice, response_format, and max_output_tokens.
• Embeddings: model, input, encoding_format: "float", and dimensions when the deployed embedding backend advertises dimension control.
• Images: model, prompt, n, plus optional size and response_format when the deployed image backend advertises them.
• Image quality, style, and user are typed pass-through OpenAI-style options when the deployed image backend supports them.
• Audio speech: model, input, voice or ref_audio plus ref_text, and optional task_type and response_format.
• Audio transcriptions: model, file, filename, optional language, optional prompt, and JSON response formats only.

The native typed helpers do not claim GA support for tool calls, structured JSON schema outputs, logprobs, seeds, service tiers, parallel tool calls, Responses state/include/reasoning controls, embedding base64 output, image streaming or partial images, audio streaming, audio translations, or direct browser API-key use. Embedding encoding_format values other than "float" and transcription response_format values other than "json" or "verbose_json" are rejected locally because they would not match the typed native SDK response objects. Unsupported OpenAI-style body parameters must fail with a clear traced 4xx gateway error.

LLM pass-through options are typed for parity with the Python SDK and OpenAI-style request shapes, but actual support depends on the deployed backend. Embedding user, TTS speed, and ASR temperature are typed pass-through options for SDK parity, but actual support depends on the deployed backend.

Text to speech

TTS deployments can expose named voices or Base/reference-audio voice cloning. Use RUNINFRA_TTS_VOICE when the deployment lists a voice or speaker. Use RUNINFRA_TTS_REF_AUDIO and RUNINFRA_TTS_REF_TEXT when the deployment expects reference-audio input.

const voice = process.env.RUNINFRA_TTS_VOICE?.trim();
const refAudio = process.env.RUNINFRA_TTS_REF_AUDIO?.trim();
const refText = process.env.RUNINFRA_TTS_REF_TEXT?.trim();
const taskType = process.env.RUNINFRA_TTS_TASK_TYPE?.trim() || "Base";

const speechVoice = voice
  ? { voice }
  : refAudio && refText
    ? { ref_audio: refAudio, ref_text: refText, task_type: taskType }
    : null;

if (!speechVoice) {
  throw new Error("Set RUNINFRA_TTS_VOICE, or RUNINFRA_TTS_REF_AUDIO and RUNINFRA_TTS_REF_TEXT.");
}

const audio = await client.audio.speech.create({
  model: "your-tts-model-id",
  input: "Hello from your optimized RunInfra endpoint.",
  ...speechVoice,
});

RunInfraAudioResponse.stream() exposes the native ReadableStream<Uint8Array> from fetch without buffering it. Use it for large TTS responses when the caller owns getReader(), cancellation, and slow-consumer backpressure. The SDK does not auto-retry or replay binary TTS streams; use arrayBuffer() or blob() when you want SDK read-timeout wrapping for a finite body.

Timeouts and retries

const apiKey = process.env.RUNINFRA_API_KEY;
if (!apiKey) throw new Error("Set RUNINFRA_API_KEY before running this snippet.");

const client = new RunInfra({
  apiKey,
  timeoutMs: 60_000,
  maxRetries: 2,
  retryBaseMs: 250,
});

The SDK retries transient transport failures and 408, 409, 429, 500, 502, 503, and 504 responses for safe GET requests. Charge-bearing POST inference requests retry only when you provide idempotencyKey, and automatic POST retries are limited to non-streaming JSON calls whose gateway responses can be replayed safely. Only responses.create() and non-streaming chat.completions.create() are currently auto-retry replay-safe. Embeddings, images, streaming calls, binary TTS responses, and multipart ASR uploads are sent once even when you provide an idempotency key. Keep maxRetries: 0 for any cost-sensitive operation whose replay behavior is not documented as safe. Automatic retries honor reasonable Retry-After values up to 60 seconds when the header is a plain integer second value or HTTP-date, then fall back to bounded exponential backoff. The SDK does not retry authentication errors, insufficient credits, or unsupported operations.

For replay-safe operations, if the gateway successfully finishes a request but the response body is too large to replay from the idempotency cache, later calls with the same idempotencyKey return idempotency_replay_unavailable without running or charging the inference again.

timeoutMs must be positive, maxRetries must be a non-negative integer, and retryBaseMs must be non-negative. Unknown per-request option keys are rejected so typos do not silently disable idempotency, tracing, timeout, or retry behavior. Invalid values throw RunInfraError with type: "invalid_request_options" before any network request is sent.

Request validation

Required request fields are validated before any network request is sent. The model must be a non-blank string, chat messages must be a non-empty array, each chat message must be an object with a non-empty role, Responses input must be a non-empty string or array, Responses input array items must be objects, JSON request bodies must be serializable and contain only finite numbers, embedding input must be a non-empty string or array of non-empty strings, TTS input and image prompts must be non-empty strings, and ASR file must be a non-empty Blob. ASR multipart filenames are validated before the FormData body is built. Invalid request values throw RunInfraError with type: "invalid_request_options" and do not reach the gateway or billing path.

Use per-request options when a call needs a shorter timeout, a trace ID, or a retry-safe idempotency key. TypeScript request interfaces are closed around typed fields, and unknown direct request fields are rejected before any network request is sent. Use extraBody in request options for deliberate JSON body extensions, such as an unsupported-parameter probe. extraBody is only accepted on JSON body requests. extraBody cannot override typed request fields and is validated before the request is sent. Custom headers are for app metadata only. They cannot override SDK-controlled headers such as Authorization, Content-Type, X-Client-Request-Id, Idempotency-Key, X-RunInfra-SDK, or X-RunInfra-SDK-Version, and they cannot set transport or credential headers such as Host, Cookie, Content-Length, Transfer-Encoding, Connection, Proxy-Authorization, Api-Key, X-API-Key, X-Auth-Token, or X-Access-Token.

await client.responses.create(
  {
    model: "llama-3.1-8b",
    input: "Summarize this incident.",
  },
  {
    clientRequestId: crypto.randomUUID(),
    idempotencyKey: crypto.randomUUID(),
    timeoutMs: 20_000,
    maxRetries: 0,
  },
);

Typed errors

The SDK exposes AuthenticationError, PermissionDeniedError, RateLimitError, InsufficientCreditsError, DeploymentError, ModelNotFoundError, RunInfraTimeoutError, RunInfraConnectionError, and RunInfraStreamParseError. UnsupportedOperationError remains exported for compatibility with older v0.1.x code, but current public helpers do not raise it. RateLimitError includes retryAfterMs when the gateway returns Retry-After. PermissionDeniedError.type preserves a specific gateway discriminator on 403 responses when one is present (for example byoc_plan_required when a workspace below the deploy tier calls a BYOC-deployed endpoint); it falls back to permission_denied. Branch on err.type instead of matching the message string. InsufficientCreditsError includes currentBalanceCents, requiredCents, and topupUrl when the gateway returns them on a 402 response, so you can render an exact top-up prompt without parsing the message. RunInfraStreamParseError includes requestId when a malformed SSE frame came from a traced gateway response. RunInfraTimeoutError also covers stalled streaming reads, stalled non-streaming JSON body reads, and stalled binary audio arrayBuffer() / blob() reads after headers arrive, and includes requestId when the response was traced. RunInfraConnectionError also covers streaming body transport failures, non-streaming JSON body transport failures, and binary audio arrayBuffer() / blob() transport failures after headers arrive, and includes requestId when the response was traced.

JSON helpers return typed response objects for the public gateway contract: ModelListResponse, ModelObject, EmbeddingResponse, ResponsesCreateResponse, ChatCompletionResponse, TranscriptionResponse, and ImageGenerationResponse. Binary TTS returns RunInfraAudioResponse, and streaming calls return RunInfraStream.

Traceability

Every request includes X-RunInfra-SDK: typescript, X-RunInfra-SDK-Version, and X-Client-Request-Id. These headers help support trace requests without changing billing or routing.

When idempotencyKey is provided, the SDK sends it as Idempotency-Key. Use a unique value for each logical retry-safe operation. Idempotency keys must be non-blank, ASCII, 255 characters or less, and must not contain secrets or personal data.

Successful JSON object responses include _request_id when the gateway returns x-request-id. Streaming responses expose the same value as stream.requestId, malformed stream frames raise RunInfraStreamParseError with that request id, and binary audio responses expose it as response.requestId. Gateway errors expose requestId, type, and, when returned by the API, OpenAI-style code and param metadata such as unsupported_parameter and dimensions. Log the request id with production errors and customer support reports.

Webhook verification

Webhook delivery management is outside the public SDK surface. The SDK includes local verification helpers for signed RunInfra webhook deliveries once you receive them in your own server. Always verify the exact raw body before parsing JSON. The RunInfra-Signature timestamp must be a non-negative integer Unix second.

import {
  WebhookVerificationError,
  constructWebhookEvent,
  verifyWebhookSignature,
} from "@runinfra/sdk";

const webhookSecret = process.env.RUNINFRA_WEBHOOK_SECRET;
if (!webhookSecret?.trim()) throw new Error("Set RUNINFRA_WEBHOOK_SECRET before verifying webhook events.");

const event = constructWebhookEvent({
  payload: rawBody,
  signatureHeader: request.headers.get("RunInfra-Signature") ?? "",
  secret: webhookSecret,
});

constructWebhookEvent verifies the signature, checks timestamp tolerance, and parses JSON. Use verifyWebhookSignature when your framework parses JSON separately and you only need to validate the raw delivery. Invalid signatures, stale timestamps, and invalid webhook JSON raise WebhookVerificationError.

OpenAI-compatible clients

OpenAI-compatible clients can use the same verified base URL:

import OpenAI from "openai";

const apiKey = process.env.RUNINFRA_API_KEY;
if (!apiKey) throw new Error("Set RUNINFRA_API_KEY before running this snippet.");

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.runinfra.ai/v1/pipe_123",
});

Voice pipelines and webhooks

Co-located voice pipelines are available through the native client.voice.pipeline.create() helper on pipeline-scoped keys. The helper posts binary audio to the pipeline-scoped /pipeline route and returns the JSON transcript / response envelope.

Webhook delivery management is handled outside the public SDK surface. Local signature verification helpers are available now.