@kwhinnery-openai/realtime-client

OpenAI Realtime API Client for JavaScript and TypeScript

This is a client library for OpenAI's Realtime API. Use it either on the server or in the browser to interact with Realtime models.

• TypeScript API reference (JSR)
• OpenAI Docs

Installation

Install from npm:

npm i @openai/realtime-client
yarn add @openai/realtime-client
pnpm add @openai/realtime-client

Install from JSR:

deno add jsr:@openai/realtime-client

Usage

This client library helps initialize a connection to the OpenAI Realtime API over either WebRTC or a WebSocket. You can then send and receive typed events over the channel you selected. WebRTC is the default connection in environments that support it.

Browser implementation

The following shows basic usage of the SDK in the browser. This code assumes that you have an API endpoint GET /session which returns an ephemeral OpenAI API token that can be used to initialize the client SDK.

import OpenAIRealtimeClient from "@openai/realtime-client";

// Client is a typed EventEmitter - takes a URL where an ephemeral key
// can be fetched from
const client = new OpenAIRealtimeClient({
  apiKeyUrl: "/session",
});

// Listen for server-sent events
client.on("session.created", (e) => {
  // First server-sent event
  console.log(e);

  // Emit client-side events
  client.send({
    type: "session.update",
    session: {
      /* session config */
    },
  });
});

// Start the Realtime session
await client.start();

Check out the event reference to see all the available client and server events.

Token server implementation

A basic Node.js server which would implement the /session endpoint would look like this (using the official OpenAI REST API client).

import OpenAI from "openai";
import Fastify from "fastify";

const client = new OpenAI();
const app = Fastify({ logger: true });

// Return session resource on the developer's server using an API token
app.get("/session", async (_req, res) => {
  const sessionConfig = {
    modalities: ["audio", "text"],
    /* ... realtime session config ... */
  };

  const sessionResponse = await client.realtime.sessions
    .create(sessionConfig)
    .asResponse();

  res.type("application/json");
  res.send(sessionResponse.body);
});

app.listen({ port: 3000 });

Middle tier implementation (Node.js or Deno)

For server-to-server use cases, you can also use a WebSocket interface and a regular OpenAI API key.

import OpenAIRealtimeClient from "@openai/realtime-client";

// reads OPENAI_API_KEY from system environment
const client = new OpenAIRealtimeClient();

client.on("session.created", (e) => {
  console.log(e);

  // Send a client event
  client.send({
    type: "session.update",
    session: {
      /* session config */
    },
  });
});

// WebSocket is the default in Node and Deno
await client.start();

Contributing

See CONTRIBUTING.md.

License

MIT