@evalguard/otel-sdk

@evalguard/otel-sdk

OpenTelemetry SDK for EvalGuard. Auto-instrument LLM calls and send traces + metrics to the EvalGuard platform over OTLP/HTTP — no collector or agent to install.

Install

npm install @evalguard/otel-sdk @opentelemetry/api

⚠️ ESM-only — requires "type": "module"

@evalguard/otel-sdk ships ES modules only ("type": "module", no CJS build). In a default CommonJS TypeScript project the imports below fail to type-check with TS1479 ("the referenced file is an ECMAScript module and cannot be imported with require").

To use this package as documented, your consuming project must be ESM:

// package.json
{ "type": "module" }

// tsconfig.json
{ "compilerOptions": { "module": "node16", "moduleResolution": "node16" } }

Staying on CommonJS? A dynamic import() works from CJS — but the await must sit inside an async function. Top-level await is an ESM-only feature, so a bare await import(...) at file scope in a CJS module is TS1309: The current file is a CommonJS module and cannot use 'await' at the top level, and running the emitted JS fails with SyntaxError: await is only valid in async functions and the top level bodies of modules. That is why this block is an async function and not two loose statements:

// ✅ compiles under module/moduleResolution "node16", no "type": "module"
async function main() {
  const { initEvalGuard } = await import("@evalguard/otel-sdk");

  const { shutdown, instrumentationReady } = initEvalGuard({
    apiKey: process.env.EVALGUARD_API_KEY!,
    projectId: "my-project-id",
    serviceName: "my-llm-app",
  });

  await instrumentationReady;

  // …then use the SDK exactly as the quick start below does.
  return shutdown;
}

void main();

// ❌ still fails — `await` at file scope in a CommonJS module
const { initEvalGuard } = await import("@evalguard/otel-sdk");
// error TS1309: The current file is a CommonJS module and cannot use
//   'await' at the top level.

Every other snippet in this README uses the static import form and top-level await, both of which need an ESM consumer.

Node.js ≥ 22.12 can also require() an ESM module directly (require(esm)), but TypeScript still type-checks the import under CJS rules, so the dynamic-import form above is the supported path.

@opentelemetry/api is a peer dependency (^1.9.1).

Quick start

import { initEvalGuard } from "@evalguard/otel-sdk";

const { shutdown, instrumentationReady } = initEvalGuard({
  apiKey: process.env.EVALGUARD_API_KEY!,
  projectId: "my-project-id",
  serviceName: "my-llm-app",
});

// Auto-instrumentation loads each SDK with `await import()`, which is the only
// mechanism that returns the module instance an ESM app holds — a dual-published
// SDK exposes a DIFFERENT class through `require`. So it finishes asynchronously.
// Clients you construct later are covered either way (shared prototypes are
// patched); await this if your very first LLM call happens immediately.
await instrumentationReady;

// From here, LLM SDK calls are auto-traced and exported to EvalGuard.

// On process exit — flush and close all providers:
await shutdown();

By default, initEvalGuard auto-instruments OpenAI, Anthropic, LiteLLM, and Google (Gemini) SDK calls. Set enableLLMInstrumentation: false to opt out and instrument manually.

instrumentationReady resolves to a per-SDK report — { name, moduleId, applied, reason }. applied is true only when a method was really replaced; it is never set because a module merely loaded. reason separates "not-installed" (an expected, silent skip) from "load-failed" (the SDK IS installed and its calls will not be traced — always warned about, never reported as "not installed").

Configuration

initEvalGuard(config) accepts (see EvalGuardConfig in src/types.ts):

Option	Type	Default	Notes
apiKey	string	—	Required. Your EvalGuard API key (eg_…).
baseUrl	string	https://evalguard.ai	EvalGuard API base URL.
projectId	string	—	Associates traces with a project.
serviceName	string	unknown-service	Resource service.name.
sampleRate	number	1.0	Head sampling ratio (0–1).
enableMetrics	boolean	false	Also collect and export metrics.
enableLLMInstrumentation	boolean	true	Auto-instrument LLM SDKs.
debug	boolean	false	Console debug logging.
batchSize	number	100	Max spans per export batch.
flushInterval	number	5000	Ms between automatic batch exports.

initEvalGuard returns { shutdown, tracerProvider, meterProvider, instrumentationReady }.

Manual instrumentation

For finer control, import the individual instrumentors and span helpers instead of relying on the auto-instrumentation:

import {
  EvalGuardSpanExporter,
  instrumentOpenAI,
  instrumentAnthropic,
  llmSpan,
} from "@evalguard/otel-sdk";

instrumentOpenAI();

const result = await llmSpan("chat", async () => {
  // ... your LLM call ...
});

All 25 instrumentors

Every instrumentor below is exported from the package root and from the @evalguard/otel-sdk/instrumentors subpath. (Through 1.0.1 the root re-exported only the first ten and there was no subpath, so the other fifteen shipped in the tarball with no supported import — fixed in 1.1.0.)

Providers	instrumentOpenAI, instrumentAnthropic, instrumentGoogle, instrumentBedrock, instrumentAzureOpenAI, instrumentGroq, instrumentMistral, instrumentCohere, instrumentDeepSeek, instrumentPerplexity, instrumentTogether, instrumentFireworks, instrumentReplicate, instrumentHuggingFace
Gateways / local runtimes	instrumentLiteLLM, instrumentVLLM, instrumentOllama
Frameworks / agents	instrumentLangChain, instrumentLangGraph, instrumentLlamaIndex, instrumentCrewAI, instrumentVercelAI, instrumentPydanticAI, instrumentOpenAIAgents, instrumentMastra

// Both of these work:
import { instrumentAzureOpenAI } from "@evalguard/otel-sdk";
import { instrumentOllama } from "@evalguard/otel-sdk/instrumentors";

initEvalGuard's auto-instrumentation applies OpenAI, Anthropic, LiteLLM and Google only. Call the others yourself.

Available typed span helpers: llmSpan, toolSpan, agentSpan, workflowSpan, embeddingSpan, retrievalSpan.

Prefer a raw OTLP exporter?

If you already run the OpenTelemetry Collector or a first-party OTLP SDK, you can skip this package and point any OTLP/HTTP exporter directly at EvalGuard's per-signal ingest endpoints. See the OpenTelemetry docs.

License

Apache-2.0. The full text ships in this package as LICENSE.