@inferagraph/openai-provider

@inferagraph/openai-provider

OpenAI provider plugin for @inferagraph/core. Includes chat (complete + stream with tool calls) and embeddings (embed).

Installation

pnpm add @inferagraph/openai-provider @inferagraph/core

Usage

import { openaiProvider } from '@inferagraph/openai-provider';
import { InferaGraph } from '@inferagraph/core/react';

<InferaGraph
  data={data}
  llm={openaiProvider({
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'gpt-4o-mini',                   // optional; default 'gpt-4o-mini'
    embeddingModel: 'text-embedding-3-small', // optional; default 'text-embedding-3-small'
  })}
/>

Configuration

Option	Description
apiKey	OpenAI API key. Server-side only. Ignored when client is provided.
model	Chat model. Default 'gpt-4o-mini'.
embeddingModel	Default embedding model. Default 'text-embedding-3-small'. Per-call override via EmbedOptions.model.
baseURL	Override the OpenAI endpoint — works for Azure OpenAI, OpenRouter, GitHub Models, or any compatible API. Ignored when client is provided.
organization	Optional OpenAI org id. Ignored when client is provided.
project	Optional OpenAI project id. Ignored when client is provided.
client	Pre-built OpenAI SDK client. When supplied, all other connection fields are ignored. Primary use case: tests / mocks.

Azure OpenAI

Use azureOpenaiProvider — it encapsulates Azure OpenAI v1 SDK construction so you never build an OpenAI / AzureOpenAI client by hand:

import { azureOpenaiProvider } from '@inferagraph/openai-provider';

azureOpenaiProvider({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT!, // e.g. https://my-resource.openai.azure.com/
  apiKey: process.env.AZURE_OPENAI_KEY!,
  deployment: 'gpt-4o',                          // chat deployment name
  embeddingDeployment: 'text-embedding-3-small', // optional; omit for chat-only
});

Option	Description
endpoint	Bare resource URL. Trailing slashes are trimmed; the factory appends /openai/v1/ for you. The v1 surface replaces dated api-version query strings entirely (Azure docs).
apiKey	Azure OpenAI API key. Server-side only.
deployment	Chat deployment name. Sent as model on chat completions.
embeddingDeployment	Embedding deployment name. Sent as model on embeddings calls. Optional — when omitted the provider has no embed capability.
client	Pre-built OpenAI SDK client. When supplied, all other connection fields are ignored. Primary use case: tests / mocks.

The factory uses the standard OpenAI class (NOT AzureOpenAI) — the v1 endpoint is fully OpenAI-compatible.

Legacy escape hatch

For OpenRouter, GitHub Models, or any other OpenAI-compatible API, the openaiProvider baseURL (or pre-built client) escape hatch still works:

openaiProvider({
  apiKey: process.env.OTHER_PROVIDER_KEY!,
  baseURL: 'https://other-openai-compatible-host/v1',
  model: 'gpt-4o',
});

For Azure specifically, prefer azureOpenaiProvider over hand-rolling openaiProvider({ client: new AzureOpenAI(...) }) — the new factory keeps Azure-specific URL math out of your codebase.

Streaming + tool calls

stream() translates the locked LLMToolDefinition[] shape into OpenAI's function-tool format and emits unified LLMStreamEvents (text / tool_call / done). Tool calls are buffered per index across deltas and flushed after the text stream closes — matching the contract's documented order.

streamMessages(messages, opts) (recommended)

stream(prompt: string) accepts a single user prompt. streamMessages(messages) accepts a structured conversation array, which unlocks:

• system role for system prompts. Tool-use-trained models heavily discount instructions delivered as user-role content; passing them under system keeps directives where the model is trained to obey them. (Better than prepending to the user message.)
• assistant role to replay prior model turns — multi-turn conversation memory, corrective-retry flows after malformed tool calls, etc.
• Multi-turn conversations as a sequence of alternating user / assistant turns following an optional leading system turn.

Signature (peer dep @inferagraph/core@^0.8.0 exports the LLMMessage / LLMRole types):

import type { LLMMessage, LLMRole } from '@inferagraph/core';

provider.streamMessages(
  messages: LLMMessage[],
  opts?: StreamOptions,
): AsyncIterable<LLMStreamEvent>;

Example — system prompt plus a 2-turn exchange:

import { openaiProvider } from '@inferagraph/openai-provider';
import type { LLMMessage } from '@inferagraph/core';

const provider = openaiProvider({
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-4o-mini',
});

const messages: LLMMessage[] = [
  { role: 'system', content: 'You are a concise assistant. Reply in one sentence.' },
  { role: 'user', content: 'Who wrote the Iliad?' },
  { role: 'assistant', content: 'Tradition attributes the Iliad to Homer.' },
  { role: 'user', content: 'And the Odyssey?' },
];

for await (const ev of provider.streamMessages!(messages)) {
  if (ev.type === 'text') process.stdout.write(ev.delta);
  if (ev.type === 'done') break;
}

The OpenAI SDK keeps system inline as the first message in the chat.completions messages array, so the contract array maps onto the SDK call essentially 1:1 — roles survive end-to-end.

Back-compat

stream(prompt) still works and is unchanged. It is internally a thin wrapper that calls streamMessages([{ role: 'user', content: prompt }]), so single-prompt behavior is identical. New consumers should prefer streamMessages whenever a system prompt or prior turns are involved.

Embeddings

embed(texts) returns Vector[] (one per input, in input order) using the configured embeddingModel. Empty inputs short-circuit to [] rather than calling the API.

License

MIT