@kwhinnery-openai/realtime-client

OpenAI Realtime API Client for JavaScript and TypeScript

This is a client library for OpenAI's Realtime API. Use it either on the server or in the browser to interact with Realtime models.

• TypeScript API reference (JSR)
• OpenAI Docs

Installation

Install from npm using the package manager of your choice:

npm i @openai/realtime-client
yarn add @openai/realtime-client
pnpm add @openai/realtime-client

Install from JSR:

deno add jsr:@openai/realtime-client
npx jsr add @openai/realtime-client

Usage

Browser implementation

The following shows basic usage of the SDK in the browser. This code assumes that you have an API endpoint GET /session which returns an ephemeral OpenAI API token that can be used to initialize the client SDK.

import OpenAIRealtimeClient from "@openai/realtime-client";

// Client is a typed EventEmitter
const client = new OpenAIRealtimeClient();

// Listen for server-sent events
client.on("session.created", (e) => {
  // First server-sent event
  console.log(e.session);

  // Emit client-side events
  client.emit("session.update", {/* event payload */});
});

// Initialize from server, assuming JSON payload is the Session resource
await client.init("/session");

Check out the event reference to see all the available client and server events.

Token server implementation

A basic Node.js server which would implement the /session endpoint would look like this (using the official OpenAI REST API client).

import OpenAI from "openai";
import Fastify from "fastify";

const client = new OpenAI();
const app = Fastify({ logger: true });

// Return session resource on the developer's server using an API token
app.get("/session", async (_req, res) => {
  const sessionConfig = {
    modalities: ["audio", "text"],
    /* ... realtime session config ... */
  };

  const sessionResponse = await client.realtime.sessions
    .create(sessionConfig)
    .asResponse();

  res.type("application/json");
  res.send(sessionResponse.body);
});

app.listen({ port: 3000 });

Middle tier implementation (Node.js or Deno)

For server-to-server use cases, you can also use a WebSocket interface and a regular OpenAI API key.

import OpenAIRealtimeClient from "@openai/realtime-client";

// Pass in your API key here
const client = new OpenAIRealtimeClient({
  apiKey: process.env.OPENAI_API_KEY,
});

client.on("session.created", (e) => {
  console.log(e.session);

  client.emit("session.update", {/* event payload */});
});

await client.init();

Contributing

See CONTRIBUTING.md.

License

MIT