@convai/experience-embed

@convai/experience-embed

A library for embedding Pixel Streaming functionality in web applications, supporting both React and vanilla JavaScript/TypeScript implementations.

Installation

npm install @convai/experience-embed

Authentication

Every embed session is authorized with a short-lived Convai auth token. Your server mints the token with your API key, and the page passes it to the embed as authToken. The API key itself never reaches the browser.

 your server                         your page                        Convai
 ───────────                         ─────────                        ──────
 POST /user/connect  ─────────────────────────────────────────────▶  mint token
   (CONVAI-API-KEY header)  ◀──── { apiAuthToken, expirationTime } ──
 GET /api/convai-token ◀──fetch──  page load
                        ──token──▶ <PixelStreamComponent authToken=…>
                                     POST viewPublishedExperience ──▶  session
                                       (API-AUTH-TOKEN header)

With a token, the page's origin does not need to be on a domain allowlist. The same code works on localhost, preview deployments and production.

1. Mint a token on your server

POST https://api.convai.com/user/connect with your API key in the CONVAI-API-KEY header. The response is { "apiAuthToken": "…", "expirationTime": "2026-10-01T10:15:00.123456" }. Tokens are valid for 60 minutes. expirationTime is in UTC but has no timezone suffix, so parse it as UTC (as below), not as local time.

A minimal Node/Express route:

// server.js: keep CONVAI_API_KEY in your server environment, never in client code
let cached = null; // { token, expiresAt }

// expirationTime is UTC without a "Z"; Date.parse would read it as local time.
const parseUtc = (t) => Date.parse(/[zZ]|[+-]\d\d:?\d\d$/.test(t) ? t : `${t}Z`);

app.get("/api/convai-token", async (req, res) => {
  // Reuse the token until 5 minutes before it expires. Minting is rate-limited
  // per API key (the limit depends on your plan), so do not mint one per page view.
  if (!cached || cached.expiresAt - Date.now() < 5 * 60 * 1000) {
    const r = await fetch("https://api.convai.com/user/connect", {
      method: "POST",
      headers: {
        "CONVAI-API-KEY": process.env.CONVAI_API_KEY,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({}),
    });
    if (!r.ok) return res.status(502).json({ error: "could not mint token" });
    const { apiAuthToken, expirationTime } = await r.json();
    cached = { token: apiAuthToken, expiresAt: parseUtc(expirationTime) };
  }
  res.json({ token: cached.token });
});

The same logic works in a Next.js route handler, a serverless function or any other backend. Put your own login or rate limiting in front of this route if the experience should not be public.

2. Pass the token to the embed

Fetch the token before you create the embed, and pass it as authToken:

const [token, setToken] = useState<string>();

useEffect(() => {
  fetch("/api/convai-token")
    .then((r) => r.json())
    .then(({ token }) => setToken(token));
}, []);

return token ? (
  <PixelStreamComponent ref={pixelStreamRef} expId="your-experience-id" authToken={token} />
) : (
  <div>Loading…</div>
);

Token rules

• Mint on a server. Never put your API key in the page, a bundle or a public repo. Anyone with the key has full API access to your account.
• The token is only needed to start a session. A stream keeps running after the token expires. Fetch a fresh token when you start a new session, for example on the next page load.
• Don't change authToken during a session. The embed treats a new authToken (like a new expId or endUserId) as a new session, and ends the current one.
• Treat the token like a password while it lives. It grants API access to your account until expirationTime. Don't log it or put it in URLs.
• A rejected token shows an error, not the allowlist message. If the token is invalid or expired, the embed shows "the Convai auth token is invalid or has expired" instead of starting a session. Mint a new one and remount the embed.
• Local development uses the same flow. Run your token route locally; http://localhost works without any allowlisting.

Legacy: domain allowlist

Embeds without authToken are still authorized by the page's domain, which Convai must add to your account's allowlist. This remains available for static sites with no backend to mint tokens, but it does not work on localhost, and every new origin (including preview URLs) needs a request to Convai. Prefer tokens for new integrations.

Step-by-Step Usage Guide

React Implementation

Step 1: Import the Component

First, import the PixelStreamComponent in your React application:

import { PixelStreamComponent } from "@convai/experience-embed";

Step 2: Set Up the Component Reference

Create a ref to access the component's methods:

import { useRef } from "react";
import { PixelStreamComponentHandles } from "@convai/experience-embed";

// Inside your component:
const pixelStreamRef = useRef<PixelStreamComponentHandles>(null);

Step 3: Available Methods

The component provides several methods through the ref:

• enableCamera(): Enables the camera for the experience
• disableCamera(): Disables the camera
• enableCharacterAudio(): Enables character audio
• disableCharacterAudio(): Disables character audio
• muteMicrophone() / unmuteMicrophone(): Mutes or unmutes the user's microphone while the conversation stays active (see Muting the microphone)
• setChatStyle(style): Changes the built-in chat / transcript appearance at runtime (see Chat transparency)
• initializeExperience(): Initializes the experience

Example usage:

const handleEnableCamera = async () => {
  await pixelStreamRef.current?.enableCamera();
};

Step 4: Create the Component

Add the PixelStreamComponent to your JSX with the required props. token is the auth token your server minted (see Authentication):

<PixelStreamComponent
  ref={pixelStreamRef}
  expId="your-experiment-id"
  authToken={token}
  endUserId="user-123"           // Required when using endUserMetadata
  endUserMetadata={{             // Optional: additional user context
    plan: "pro",
    region: "us-east",
  }}
  InitialScreen={<div>Loading your experience...</div>}
  chatStyle={{ backgroundOpacity: 0.3 }} // Optional: more transparent chat backplate
  onIframeLoaded={() => console.log("stream page loaded")}
  onCharacterMessage={(msg) => console.log(msg.name, msg.message)}
/>

Step 5: Customize the Initial Screen (Optional)

You can create a custom loading screen:

const CustomLoadingScreen = () => (
  <div
    style={{
      width: "100%",
      height: "100%",
      display: "flex",
      alignItems: "center",
      justifyContent: "center",
      background: "#000",
      color: "#fff",
    }}
  >
    <h2>Loading your experience...</h2>
  </div>
);

// Use it in your component:
<PixelStreamComponent
  ref={pixelStreamRef}
  expId="your-experiment-id"
  authToken={token}
  InitialScreen={<CustomLoadingScreen />}
/>;

Step 6: Add Loading Screen for Embed (Optional)

Show a loading overlay until the embed sends isLoading: false:

<PixelStreamComponent
  ref={pixelStreamRef}
  expId="your-experiment-id"
  authToken={token}
  LoadingScreenComponent={<div>Preparing experience...</div>}
/>

Core (Vanilla TypeScript) Implementation

Step 1: Import the Client

Import the PixelStreamClient in your TypeScript file:

import { PixelStreamClient } from "@convai/experience-embed/core";

Step 2: Create Container Element

Create a container element in your HTML:

<div id="pixel-stream-container" style="width: 100%; height: 600px;"></div>

Step 3: Initialize the Client

Fetch an auth token from your server (see Authentication), then create a new instance of PixelStreamClient:

const { token } = await fetch("/api/convai-token").then((r) => r.json());

const container = document.getElementById("pixel-stream-container");
if (container) {
  const pixelStream = new PixelStreamClient({
    container: container,
    expId: "your-experiment-id",
    authToken: token,
    endUserId: "user-123",       // Required when using endUserMetadata
    endUserMetadata: {           // Optional: additional user context
      plan: "pro",
      region: "us-east",
    },
    InitialScreen: document.createElement("div"),
    LoadingScreenComponent: document.createElement("div"),
    chatStyle: { backgroundOpacity: 0.3 }, // Optional: more transparent chat backplate
    onIframeLoaded: () => console.log("stream page loaded"),
    onCharacterMessage: (msg) => console.log(msg.name, msg.message),
  });
}

Step 4: Available Methods

The client provides the following methods:

// Enable camera
pixelStream.enableCamera().then((success) => {
  console.log("Camera enabled:", success);
});

// Disable camera
pixelStream.disableCamera().then((success) => {
  console.log("Camera disabled:", success);
});

// Enable character audio
pixelStream.enableCharacterAudio();

// Disable character audio
pixelStream.disableCharacterAudio();

// Initialize experience
pixelStream.initializeExperience();

// Make the chat / transcript backplate more transparent
pixelStream.setChatStyle({ backgroundOpacity: 0.3 });

CDN Implementation

You can use the @convai/experience-embed package directly in your HTML without any build tools by loading it from a CDN.

Step 1: Include the UMD Script

Include the library via CDN in your HTML:

<script src="https://unpkg.com/@convai/experience-embed/dist/core/convai-embed.umd.js"></script>

Step 2: Create Container Element

Create a container element in your HTML:

<div id="pixel-stream-container" style="width: 100%; height: 600px;"></div>

Step 3: Initialize the Client

Fetch an auth token from your server (see Authentication), then create a new instance of PixelStreamClient:

<script type="module">
  const { token } = await fetch("/api/convai-token").then((r) => r.json());
  const container = document.getElementById("pixel-stream-container");

  const customInitialScreen = document.createElement("div");
  customInitialScreen.style.cssText = `
    width: 100%;
    height: 100%;
    position: absolute;
    top: 0;
    left: 0;
    z-index: 10;
    display: flex;
    align-items: center;
    justify-content: center;
    background: rgba(0, 0, 0, 0.8);
    color: white;
    font-family: Arial, sans-serif;
  `;
  customInitialScreen.innerHTML = `
    <div style="text-align: center;">
      <h2>Welcome to the Experience</h2>
      <p>Click anywhere to start</p>
    </div>
  `;

  const pixelStream = new PixelStreamCore.PixelStreamClient({
    expId: "your-experiment-id",
    container: container,
    authToken: token,
    endUserId: "user-123",       // Required when using endUserMetadata
    endUserMetadata: {           // Optional: additional user context
      plan: "pro",
      region: "us-east",
    },
    InitialScreen: customInitialScreen,
  });

  // Make it available globally for debugging if needed
  window.pixelStream = pixelStream;
</script>

Step 4: Available Methods

Once initialized, the following methods are available on the pixelStream object:

initializeExperience() Starts the pixel streaming session:

pixelStream.initializeExperience();

enableCamera() Requests access to the user’s camera and sends the video stream to the experience:

pixelStream.enableCamera().then((success) => {
  console.log("Camera enabled:", success);
});

disableCamera() Stops the camera stream:

pixelStream.disableCamera().then((success) => {
  console.log("Camera disabled:", success);
});

enableCharacterAudio() Enables audio from the character (from Unreal/Convai):

pixelStream.enableCharacterAudio();

disableCharacterAudio() Disables audio from the character (from Unreal/Convai):

pixelStream.disableCharacterAudio();

muteMicrophone() / unmuteMicrophone() Mutes or unmutes the user's microphone while the conversation stays active (see Muting the microphone):

pixelStream.muteMicrophone();
pixelStream.unmuteMicrophone();

setChatStyle(style) Changes the built-in chat / transcript appearance, applied live (see Chat transparency):

pixelStream.setChatStyle({ backgroundOpacity: 0.3 });

Step 5: Add Interactive Buttons (Optional)

You can add buttons in your HTML to trigger these methods:

<div class="controls">
  <button onclick="pixelStream.initializeExperience()">Start Experience</button>
  <button onclick="pixelStream.enableCamera()">Enable Camera</button>
  <button onclick="pixelStream.disableCamera()">Disable Camera</button>
  <button onclick="pixelStream.enableCharacterAudio()">Enable Audio</button>
  <button onclick="pixelStream.disableCharacterAudio()">Disable Audio</button>
</div>

API Reference

PixelStreamComponent Props

Prop	Type	Required	Description
expId	string	Yes	Your experiment ID
endUserId	string	No*	User identifier for tracking and long-term memory
authToken	string	Recommended	Auth token minted by your server; see Authentication. Without it the page's domain must be allowlisted
endUserMetadata	Record<string, any>	No	Additional user context (requires endUserId)
InitialScreen	React.ReactNode	No	Custom splash screen shown before experience starts
LoadingScreenComponent	React.ReactNode	No	Loading overlay shown until embed sends isLoading: false
serviceUrls	object	No	Custom service URLs configuration
serviceUrls.sessionFetch	string	No	Custom URL for fetching the session
serviceUrls.pixelStreamBase	string	No	Custom base URL for the pixel stream
chatStyle	ChatStyle	No	Built-in chat appearance, e.g. { backgroundOpacity: 0.3 } (see Chat transparency)

* endUserId is required if you want to pass endUserMetadata. TypeScript will enforce this at compile time.

endUserId is sent to Convai when the stream session is allocated and is bound to that session for its lifetime, so session history and long-term memory attribute the conversation to this user. Set it before the component mounts (or the client is constructed); changing it afterwards allocates a new session. It must be a non-empty string of at most 512 characters.

PixelStreamClient Options

Option	Type	Required	Description
container	HTMLElement	Yes	The container element for the stream
expId	string	Yes	Your experiment ID
endUserId	string	No*	User identifier for tracking and long-term memory
authToken	string	Recommended	Auth token minted by your server; see Authentication. Without it the page's domain must be allowlisted
endUserMetadata	Record<string, any>	No	Additional user context (requires endUserId)
InitialScreen	HTMLElement	No	Custom splash screen shown before experience starts
LoadingScreenComponent	HTMLElement	No	Loading overlay shown until embed sends isLoading: false
serviceUrls	object	No	Custom service URLs configuration
serviceUrls.sessionFetch	string	No	Custom URL for fetching the session
serviceUrls.pixelStreamBase	string	No	Custom base URL for the pixel stream
chatStyle	ChatStyle	No	Built-in chat appearance, e.g. { backgroundOpacity: 0.3 } (see Chat transparency)

* endUserId is required if you want to pass endUserMetadata. TypeScript will enforce this at compile time.

endUserId is sent to Convai when the stream session is allocated and is bound to that session for its lifetime, so session history and long-term memory attribute the conversation to this user. Set it before the component mounts (or the client is constructed); changing it afterwards allocates a new session. It must be a non-empty string of at most 512 characters.

Callbacks

Pass these as props on PixelStreamComponent or as options on PixelStreamClient. Each one fires when the stream page sends the matching event.

Callback	Fires when	Payload
onIframeLoaded	The stream page has loaded in the iframe. Once per session.	null
onCharacterMessage	The user or the character says something. Character replies arrive in chunks.	{ name, message, endofmessage, sentenceready, player }
onAfk	The stream disconnects or the user goes idle	a status string, e.g. "Disconnected"
onCameraStatus	The webcam turns on or off	boolean
onCameraStream	The experience asks to start or stop the webcam	boolean
onUserUpdated	The experience updates the user's details	as sent by the experience
onConfigChanged	The experience changes its configuration	as sent by the experience
onMicStatus	The stream page applied a muteMicrophone() / unmuteMicrophone() call	{ muted: boolean }

onCharacterMessage payload fields: name is who is speaking ("User" or the character's name), message is the text of this chunk, player is true for the user's own lines, and endofmessage is true on the last chunk of a reply.

Muting the microphone

For a hands-free experience (Conversation Mode: Handsfree), use muteMicrophone() and unmuteMicrophone() for a mute button. They stop and restart the user's outgoing microphone audio; the session and conversation stay active. Do not use endVoiceInteraction() for this: in hands-free mode the character keeps hearing the user after it.

const [muted, setMuted] = useState(false);

<PixelStreamComponent
  ref={pixelStreamRef}
  expId="your-exp-id"
  onMicStatus={({ muted }) => setMuted(muted)}
/>

<button
  onClick={() =>
    muted
      ? pixelStreamRef.current?.unmuteMicrophone()
      : pixelStreamRef.current?.muteMicrophone()
  }
>
  {muted ? "Unmute" : "Mute"}
</button>

The mute survives stream reconnects. onMicStatus fires whenever the mic state changes, and also when the stream connects, with the current state, so your button can start in the right state. If the experience shows its built-in chat, the mic starts muted (the chat's own mic button and your calls share one state). If onMicStatus never fires, the stream page predates this feature.

Chat transparency

The built-in chat / transcript window draws its text, input field and mic button on dark backplates. Lower chatStyle.backgroundOpacity (0 to 1, default 0.7) to let the experience's background show through; the input field and mic button (default 0.8) scale by the same amount:

<PixelStreamComponent expId="your-exp-id" chatStyle={{ backgroundOpacity: 0.3 }} />

const pixelStream = new PixelStreamClient({
  expId: "your-exp-id",
  container,
  chatStyle: { backgroundOpacity: 0.3 },
});
pixelStream.setChatStyle({ backgroundOpacity: 0 }); // fully transparent, applied live

backgroundOpacity	Chat panel	Input field and mic button
0.7 (default)	0.7	0.8
0.3	0.3	0.34
0	clear	clear

• Changing the prop, or calling setChatStyle(), applies live. With setChatStyle(), fields you leave out keep their current value.
• Values outside 0 to 1 are clamped; anything that is not a number is ignored.
• Below 0.5 the stream page keeps the input usable: it outlines the input field and mic button, lightens the placeholder text, and fades the top of the message list instead of cutting messages off.
• In hands-free mode the transcript plates scale by the same amount.
• 0 removes the backplates entirely, and message text can be hard to read over bright scenes. Values around 0.3 work well in most scenes.
• Stream pages that predate this option ignore it, so the chat keeps its default look.

Changelog

0.7.2

Licensing only. No code changes.

• The package is licensed under the Apache License 2.0, like @convai/web-sdk. Earlier versions declared no license, which npm shows as none. The LICENSE file now ships in the package.

0.7.1

Packaging fixes. No API changes.

• import { PixelStreamClient } from '@convai/experience-embed/core' now works. The package ships an ES build of the client, so the /core entry point resolves in bundlers (Vite, webpack, esbuild) and in Node, not only through a <script> tag. The README previously showed this import from the root entry point, which exports the React component instead.
• require('@convai/experience-embed/core') now returns the client. It resolved to the UMD file, which this package's "type": "module" made Node parse as ESM, so the call returned an empty object.
• The types paths in package.json now match the directories on disk (dist/types/React, dist/types/Core). The lowercase paths resolved only on case-insensitive filesystems, so types went missing on Linux.
• The core bundle no longer contains React, which it pulled in through a barrel import. The UMD build is 46% smaller (115 KB to 61 KB).

The UMD build keeps its name and URL, dist/core/convai-embed.umd.js, and still defines the global PixelStreamCore.

0.7.0

• authToken authorizes the session with a server-minted Convai auth token, so the embedding domain no longer needs to be allowlisted (localhost included). This is now the recommended setup; see Authentication.
• When a session started with authToken fails, the error screen now says the token is invalid or expired (or that the experience couldn't start), instead of asking for the domain to be whitelisted. Sessions without a token keep the whitelist message.

0.6.2

• chatStyle option ({ backgroundOpacity }) and setChatStyle() control how transparent the built-in chat / transcript window is, on both PixelStreamComponent and PixelStreamClient. It covers the chat panel, the input field and mic button, and the hands-free transcript. See Chat transparency.
• ChatStyle is exported as a type from both entry points.

0.6.1

Use this instead of 0.6.0. 0.6.0 was published only under the beta tag, from a build that also contained the not-yet-released authToken option.

• muteMicrophone() / unmuteMicrophone() mute the user's outgoing microphone audio without ending the conversation, on both PixelStreamComponent and PixelStreamClient.
• onMicStatus reports { muted } when the stream connects and on every mute change, including changes made with the built-in chat's mic button.
• Removed callbacks that could never fire, because the stream page never sends their messages: onCharInfo (deprecated in 0.5.2), onXConvai, onClientMessage, onMicMessage, onConnectCall, onCharacterAudio, onDashboardRedirect, onToggleCamera and onIframeState. Passing one of them is now a TypeScript error; delete it, as it was never called.

0.5.2

• Iframe callbacks now fire. Before, the React component only delivered onCharacterMessage, and the vanilla PixelStreamClient delivered none.
• onIframeLoaded fires once per session.
• PixelStreamClient accepts onIframeLoaded, onAfk, onCameraStatus, onCameraStream, onUserUpdated and onConfigChanged.
• onCharInfo is deprecated.

0.5.1

• endUserId is sent when the stream session is created, so sessions, chat history and long-term memory are attributed to the end user.

License

Licensed under the Apache License 2.0. Copyright 2025 Convai.