Generic invoke - POST to any function ID with type-safe I/O
Async generator responses - Yields request ID, status updates, and final result
Configurable polling - Interval, max attempts, poll-seconds header
Asset management - Create, upload, delete lifecycle
Queue depth queries - Check function queue status
Automatic token refresh - OAuth2 client credentials flow
BullMQ integration - Resumable job processing with S3 storage and zip/base64 extraction

Installation

bun add nvcf-client

Usage

Basic Usage

import { NVCFClient } from "nvcf-client";

const client = new NVCFClient({
  auth: { type: "token", token: process.env.NVCF_TOKEN },
});

// Simple await
const result = await client.invokeAndWait<MyInput, MyOutput>({
  functionId: "abc-123",
  body: { prompt: "hello" },
});

Async Generator for Progress Tracking

for await (const event of client.invoke<MyInput, MyOutput>({
  functionId: "abc-123",
  body: { prompt: "hello" },
})) {
  switch (event.type) {
    case "request_id":
      console.log("Started:", event.requestId);
      break;
    case "status":
      console.log("Status:", event.status);
      break;
    case "fulfilled":
      console.log("Done:", event.data);
      break;
  }
}

OAuth2 Client Credentials

const client = new NVCFClient({
  auth: {
    type: "client_credentials",
    authUrl: "https://auth.nvidia.com/oauth2/token",
    clientId: process.env.CLIENT_ID,
    clientSecret: process.env.CLIENT_SECRET,
    tokenRefreshBufferSeconds: 60,
  },
});

BullMQ Integration

Use processNVCFStep to build resumable workers with crash recovery. Each step tracks its state (pending → submitted → polling → completed) and can be resumed by another worker if the original dies.

The storage configuration supports:

Zip responses: Extracts response.json for output, extracts files by zipFilename
Direct JSON responses: Extracts base64 via user-provided getBase64 function

import { Worker } from "bullmq";
import { S3Client } from "@aws-sdk/client-s3";
import { NVCFClient, processNVCFStep, NVCFStepState } from "nvcf-client";

type ImageOutput = { image_output: string; prompt: string };

type MyJobData = {
  prompt: string;
  steps?: {
    generate?: NVCFStepState<ImageOutput>;
  };
};

const client = new NVCFClient({ auth: { type: "token", token: "..." } });
const s3 = new S3Client({ region: "us-east-1" });

const worker = new Worker<MyJobData>("my-queue", async (job) => {
  const generateState = job.data.steps?.generate ?? { phase: "pending" };

  const result = await processNVCFStep<{ prompt: string }, ImageOutput>(
    client,
    {
      functionId: "sdxl",
      body: { prompt: job.data.prompt },
      assetReferences: undefined,
    },
    generateState,
    {
      onStateChange: async (state) => {
        await job.updateData({
          ...job.data,
          steps: { ...job.data.steps, generate: state },
        });
      },
      storage: {
        s3,
        s3Config: { bucket: "outputs", keyPrefix: "images/" },
        outputFileFields: [
          {
            name: "image",
            zipFilename: "image.jpg",  // for zip responses
            getBase64: (output) => output.image_output,  // for JSON responses
          },
        ],
      },
    }
  );

  if (result.status === "success") {
    console.log("Output:", result.output);
    console.log("Image URL:", result.files?.image);
  }

  return result;
}, { connection: { host: "localhost" } });

Output File Field Configuration

Each outputFileFields entry specifies a file to extract:

type OutputFileField<TOutput> = {
  // Key in the result files map
  name: string;

  // Filename to look for in zip (also used to infer content-type)
  zipFilename: string;

  // S3 key prefix for this field (overrides s3Config.keyPrefix)
  s3KeyPrefix?: string;

  // For JSON responses: function to extract base64 string from output
  getBase64?: (output: TOutput) => string | undefined;
};

Configuration

type NVCFConfig = {
  baseUrl?: string; // default: https://api.nvcf.nvidia.com
  auth:
    | { type: "token"; token: string }
    | {
        type: "client_credentials";
        authUrl: string;
        clientId: string;
        clientSecret: string;
        tokenRefreshBufferSeconds?: number;
      };
  polling?: {
    intervalMs?: number; // default: 1000
    maxAttempts?: number; // default: 60
    pollSecondsHeader?: number; // default: 60
  };
  retry?: {
    maxAttempts?: number; // default: 3
    baseDelayMs?: number; // default: 1000
    maxDelayMs?: number; // default: 10000
  };
  timeoutMs?: number; // default: 30000
};