@memberjunction/ai

Back to AI Framework Overview

@memberjunction/ai

Core abstractions and base classes for the MemberJunction AI Framework. This package defines provider-agnostic interfaces for Large Language Models, embeddings, image generation, audio, video, reranking, and more.

Zero MemberJunction dependencies beyond @memberjunction/global (which itself has no transitive dependencies). This package works in any TypeScript or JavaScript project -- no database, no metadata layer, no MJ runtime required.

Installation

npm install @memberjunction/ai

Base Classes

Every AI capability is represented by an abstract base class. Provider packages (OpenAI, Anthropic, Gemini, etc.) extend these to implement the actual API calls.

Class	Purpose	Key Methods
BaseLLM	Chat completions (text generation)	ChatCompletion(), ChatCompletions() (parallel batch), GetFileCapabilities()
BaseEmbeddings	Text & multimodal embeddings	EmbedText(), EmbedTexts(), EmbedContent(), GetFileCapabilities()
BaseImageGenerator	Image generation, editing, variations	GenerateImage(), EditImage(), CreateVariation()
BaseTextToSpeech	Text-to-speech (TTS models)	CreateSpeech(), GetVoices(), GetModels(), GetPronounciationDictionaries()
BaseSpeechToText	Speech-to-text (Speech to Text models)	SpeechToText(), GetModels(); oversized audio via TranscribeAudioWithSplitting()
BaseAudioGenerator	Deprecated. Both of the above on one class. Kept, and still working, for existing drivers and callers; it implements both new classes	CreateSpeech(), SpeechToText(), GetVoices()
BaseVideoGenerator	Avatar video generation (Video models)	CreateAvatarVideo(), GetAvatars()
BaseReranker	Document reranking for retrieval	Rerank()
BaseDecision	Typed decisions (Likelihood, Choice, Score) with a probability per answer	Decide()
BaseRealtimeModel	Live, full-duplex, tool-calling realtime sessions (voice)	StartSession(), CreateClientSession()

All inherit from BaseModel, which manages API key storage and provides the @RegisterClass integration point.

One additional realtime primitive lives here that is not a BaseModel capability: BaseRealtimeChannelServer — the server half of the realtime interactive-channel plugin contract (MJ: AI Agent Channels.ServerPluginClass), mirroring the client half (BaseRealtimeChannelClient in the Angular conversations package). Concrete plugins register via @RegisterClass(BaseRealtimeChannelServer, '<key>') and are resolved per session by RealtimeChannelServerHost in @memberjunction/ai-agents. See guides/REALTIME_CO_AGENTS_GUIDE.md §5.

Type Definitions

Chat Types

Type	Description
ChatParams	Full parameter set for chat requests: messages, model, temperature, topP, topK, streaming, effort level, response format, and more
ChatResult	Completion result with choices, token usage, cost tracking, and timing
ChatMessage	Single message with role, content (text or multimodal blocks), and optional metadata
ChatMessageContentBlock	Multimodal content: text, image (base64/URL), video, audio, or file
StreamingChatCallbacks	Callbacks for real-time streaming: OnContent, OnComplete, OnError
ParallelChatCompletionsCallbacks	Callbacks for batch parallel completions
ChatMessageRole	Enum: system, user, assistant
VolatileStateMessageMetadata / TrailingVolatileStateSplit	The one metadata flag (volatileState) drivers and the agent layer agree on for a per-request trailing message, and the { head, tail } split BaseLLM.splitTrailingVolatileState returns so a driver can place its cache boundary before the volatile tail (protected seam: isVolatileStateMessage, trailingVolatileStateIndex, splitTrailingVolatileState)

Embedding Types

Type	Description
EmbedTextParams / EmbedTextResult	Single text embedding request and response
EmbedTextsParams / EmbedTextsResult	Batch text embedding request and response
EmbedContentParams / EmbedContentResult	Multimodal embedding request (text and/or interleaved media blocks fused into one vector) and response. EmbedContent() is non-abstract — it defaults to EmbedText for text-only content; multimodal providers override it. Supported media is declared per provider via GetFileCapabilities()

Other Types

Type	Description
ImageGenerationParams / ImageGenerationResult	Image generation parameters and results
SummarizeParams / SummarizeResult	Text summarization (deprecated)
ClassifyParams / ClassifyResult	Text classification (deprecated)
RerankParams / RerankResult	Document reranking
DecisionParams / DecisionResult	Typed decision parameters and structured results with probabilities
ModelUsage	Token counts and cost tracking (prompt tokens, completion tokens, total cost, currency)
AIModelConfiguration / LLMConfigurationSettings / IsPrefixPromptCache	The model catalog's ModelConfiguration JSONType bag (Model Types < Models < the vendor's Configuration.ModelDefaults (AIVendorConfiguration, parsed by ParseVendorConfiguration) < Model Vendors, resolved by ResolveEffectiveModelConfiguration). LLMConfigurationSettings.PrefixPromptCache (read via IsPrefixPromptCache) says whether a serving path's prompt cache is an exact byte-prefix match (OpenAI, xAI — set once under the vendor row's Configuration.ModelDefaults, inherited by every model it serves) rather than block/segment based, which decides whether the agent layer appends or replaces its trailing runtime-state message
BaseResult	Common result base with success flag, timing, and error info
FileCapabilities	Declares which non-text inputs a provider accepts: SupportedMimeTypes (e.g. image/png, audio/mp3, supports image/* wildcards), MaxFileSize, MaxFilesPerRequest, HasFileAPI. Returned by GetFileCapabilities() on BaseLLM (file inputs to chat) and BaseEmbeddings (media inputs to EmbedContent); null means text-only

Utilities

Export	Description
AIAPIKeys / GetAIAPIKey()	API key resolution from environment variables (AI_VENDOR_API_KEY__<DRIVER>) with optional runtime overrides
ErrorAnalyzer	Classifies provider errors into structured types with severity, retry hints, and failover recommendations
AIErrorInfo / AIErrorType	Structured error types: rate limit, authentication, context length, content filter, etc.
serializeMessageContent() / deserializeMessageContent()	Content block serialization for database storage
parseBase64DataUrl() / createBase64DataUrl()	Base64 data URL utilities

Usage Examples

Chat Completion

import { ChatParams, ChatMessageRole } from "@memberjunction/ai";
import { OpenAILLM } from "@memberjunction/ai-openai";

const llm = new OpenAILLM("your-api-key");

const result = await llm.ChatCompletion({
    model: "gpt-4.1",
    messages: [
        { role: ChatMessageRole.system, content: "You are a helpful assistant." },
        { role: ChatMessageRole.user, content: "What is the capital of France?" },
    ],
    temperature: 0.7,
    maxOutputTokens: 500,
});

console.log(result.data.choices[0].message.content);

Streaming

await llm.ChatCompletion({
    model: "gpt-4.1",
    messages: [{ role: ChatMessageRole.user, content: "Explain quantum computing." }],
    streaming: true,
    streamingCallbacks: {
        OnContent: (chunk, isComplete) => process.stdout.write(chunk),
        OnComplete: (result) => console.log("\nDone!"),
        OnError: (error) => console.error("Stream error:", error),
    },
});

Parallel Completions

const paramSets = [
    { ...base, temperature: 0.3 },
    { ...base, temperature: 0.7 },
    { ...base, temperature: 1.0 },
];

const results = await llm.ChatCompletions(paramSets, {
    OnCompletion: (result, index) => console.log(`Completion ${index} done`),
    OnAllCompleted: (results) => console.log(`All ${results.length} complete`),
});

Multimodal Content

const result = await llm.ChatCompletion({
    model: "gpt-4.1",
    messages: [{
        role: ChatMessageRole.user,
        content: [
            { type: "text", content: "What is in this image?" },
            { type: "image_url", content: "data:image/png;base64,..." },
        ],
    }],
});

Text Embeddings

import { OpenAIEmbedding } from "@memberjunction/ai-openai";

const embedder = new OpenAIEmbedding("your-api-key");
const result = await embedder.EmbedText({
    model: "text-embedding-3-small",
    text: "Sample text to embed",
});
console.log(`Dimensions: ${result.vector.length}`);

API Key Resolution

import { GetAIAPIKey } from "@memberjunction/ai";

// Reads AI_VENDOR_API_KEY__OPENAILLM from environment
const key = GetAIAPIKey("OpenAILLM");

// With runtime override
const key2 = GetAIAPIKey("AnthropicLLM", [
    { driverClass: "AnthropicLLM", apiKey: "sk-ant-..." },
]);

Implementing a New Provider

Extend the base class for the capability you want to support:

import { BaseLLM, ChatParams, ChatResult, ClassifyParams, ClassifyResult, SummarizeParams, SummarizeResult } from "@memberjunction/ai";
import { RegisterClass } from "@memberjunction/global";

@RegisterClass(BaseLLM, "MyProviderLLM")
export class MyProviderLLM extends BaseLLM {
    // Required: implement non-streaming chat
    protected async nonStreamingChatCompletion(params: ChatParams): Promise<ChatResult> {
        // Your API call here
    }

    // Required by BaseLLM but deprecated: new drivers should throw a "not supported" error
    public async ClassifyText(params: ClassifyParams): Promise<ClassifyResult> { /* ... */ }

    // Required by BaseLLM but deprecated: new drivers should throw a "not supported" error
    public async SummarizeText(params: SummarizeParams): Promise<SummarizeResult> { /* ... */ }

    // Optional: enable streaming by overriding these
    public get SupportsStreaming(): boolean { return true; }
    protected async createStreamingRequest(params: ChatParams): Promise<AsyncIterable<unknown>> { /* ... */ }
    protected processStreamingChunk(chunk: unknown): { content: string } { /* ... */ }
    protected finalizeStreamingResponse(content: string, lastChunk: unknown, usage: unknown): ChatResult { /* ... */ }
}

See the full provider list for working examples across 25+ implementations.

Dependencies

Package	Purpose
@memberjunction/global	Class factory and global utilities (zero transitive dependencies)
dotenv	Environment variable loading
rxjs	Reactive extensions (internal use)

Related Packages

• AI Framework Overview -- Architecture, provider matrix, and quick start
• Providers -- All 25+ provider implementations
• Prompts -- MJ-integrated prompt template engine
• Agents -- Agent execution framework