ai-patterns

ai-patterns

Battle-tested TypeScript patterns for building rock-solid AI applications.

We provide developers with battle-tested tools for resilient AI workflows: retry logic, circuit breakers, rate limiting, human-in-the-loop escalation, and more — all with complete type safety and composability. Inspired by Vercel AI SDK's developer experience.

Features

✨ Battle-Tested Patterns - Retry, Circuit Breaker, Timeout, Rate Limiter, Fallback, Cache, Debounce, Throttle, Bulkhead, A/B Testing, Cost Tracking, Prompt Versioning, Response Validation, Context Window Management, Reflection Loop, and more 🎨 Elegant Composition - Compose patterns together for complex workflows 🔒 Type-Safe - Full TypeScript support with generics and strict mode 🧩 Composable - Patterns work together seamlessly for robust workflows 📊 Observable - Built-in lifecycle callbacks for monitoring and debugging 🪶 Lightweight - Zero dependencies, minimal overhead ⚡ Production-Ready - Build solid AI applications with confidence 🎯 Developer-Friendly - Inspired by Vercel AI SDK's excellent DX 💰 Cost Control - Track and control AI spending in real-time 🧪 Experimentation - A/B test prompts and models to optimize performance

Installation

npm install ai-patterns
# or
yarn add ai-patterns
# or
pnpm add ai-patterns

Quick Start

Simple Retry

import { retry } from 'ai-patterns';

// Retry any async function
const result = await retry({
  execute: () => fetch('https://api.example.com/data'),
  maxAttempts: 3
});

console.log(result.value);

With Vercel AI SDK

import { retry } from 'ai-patterns';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await retry({
  execute: async () => {
    const { text } = await generateText({
      model: openai('gpt-4-turbo'),
      prompt: 'Explain quantum computing',
      maxRetries: 0 // Disable Vercel's built-in retry
    });
    return text;
  },
  maxAttempts: 3
});

console.log(result.value);

💡 Note: While Vercel AI SDK has built-in retry (maxRetries: 2), ai-patterns gives you more flexibility:

• 🎛️ Custom backoff strategies (exponential, linear, fixed)
• 📊 Detailed observability (attempts, delays, errors)
• 🔄 Cross-provider fallback (OpenAI → Claude → Gemini)
• 🎯 Advanced retry logic (conditional, circuit breakers)

Why ai-patterns?

Building AI applications? You're probably facing these challenges:

❌ Copy-pasting retry logic across every API call ❌ No circuit breakers — one API failure brings down your entire app ❌ Constantly hitting rate limits with no systematic handling ❌ No human oversight for edge cases that need review

With ai-patterns:

✅ Battle-tested patterns ready to use out of the box ✅ Compose like Lego blocks — combine patterns seamlessly ✅ Full type safety — catch errors at compile time ✅ Zero dependencies — lightweight and production-ready

Before ai-patterns:

// 50+ lines of retry logic with exponential backoff,
// jitter, error classification, timeout handling...
let attempt = 0;
const maxAttempts = 3;
while (attempt < maxAttempts) {
  try {
    // ... complex retry logic
  } catch (error) {
    // ... backoff calculation
    // ... error handling
  }
}

After ai-patterns:

const result = await retry({
  execute: () => callAPI(),
  maxAttempts: 3
});

That's it. Simple, reliable, production-ready.

Advanced Usage

Stateful Patterns

Use defineCircuitBreaker and defineRateLimiter for patterns that maintain state:

const breaker = defineCircuitBreaker({
  execute: (prompt: string) => callAPI(prompt),
  failureThreshold: 5,
  resetTimeout: 60000
});

// Reuse the same instance across calls
await breaker('First call');
await breaker('Second call');
console.log(breaker.getState()); // Check circuit state

Pattern Composition

Compose patterns together for robust workflows using the compose() function:

import { compose, withRetry, withTimeout, withFallback } from 'ai-patterns';

// Create a reusable composed function
const robustAI = compose<string, string>([
  withFallback({ fallback: () => "Sorry, service unavailable" }),
  withTimeout({ duration: 10000 }),
  withRetry({
    maxAttempts: 3,
    backoffStrategy: "exponential",
  })
]);

// Use it anywhere
const result = await robustAI(callAI, "Explain quantum computing");

Tip: You can also nest patterns directly if you prefer explicit control flow.

For advanced composition strategies:

• Composition Guide →
• Production Examples →

Patterns

Core Patterns

Pattern	Description	Use Case	Docs
compose	Functional pattern composition	Complex AI pipelines	📖
retry	Automatic retry with exponential backoff	Unstable APIs, network issues	📖
timeout	Time limits with AbortSignal support	Long-running operations	📖
fallback	Execute alternatives on failure	Multi-provider failover	📖
defineCircuitBreaker	Protect against failing services	External API calls	📖
defineRateLimiter	Control request throughput	API rate limiting	📖

Advanced Patterns

Pattern	Description	Use Case	Docs
memoize	Cache function results with TTL	Response caching	📖
defineDebounce	Delay execution until silence period	User input handling	📖
defineThrottle	Limit execution frequency	API call throttling	📖
defineBulkhead	Isolate resources with concurrency limits	Resource isolation	📖
deadLetterQueue	Handle failed operations	Error recovery	📖

Orchestration Patterns

Pattern	Description	Use Case	Docs
fanOut	Parallel processing with concurrency control	Batch operations	📖
saga	Distributed transactions with compensation	Multi-step workflows	📖
conditionalBranch	Route based on conditions	Dynamic workflow routing	📖

AI-Specific Patterns

Pattern	Description	Use Case	Docs
humanInTheLoop	AI → Human escalation	Content moderation	📖
smartContextWindow	Manage context token limits automatically	Long conversations, chat apps	📖
reflectionLoop	AI self-critique and iterative improvement	High-quality content generation	📖
idempotency	Prevent duplicate operations	Payment processing	📖

Experimentation & Monitoring

Pattern	Description	Use Case	Docs
abTest	Test multiple variants simultaneously	Prompt optimization, model selection	📖
costTracking	Monitor and control AI spending	Budget management, cost optimization	📖
versionedPrompt	Manage prompt versions with rollback	Prompt experimentation, gradual rollout	📖
validateResponse	Validate AI responses with auto-retry	Quality assurance, business rules	📖

Pattern Examples

Robust API Call

import { retry, timeout } from 'ai-patterns';

const result = await retry({
  execute: async () => {
    return await timeout({
      execute: () => fetch('https://api.example.com/data'),
      timeoutMs: 5000
    });
  },
  maxAttempts: 3
});

AI Agent with Fallback

import { fallback } from 'ai-patterns';
import { generateText } from 'ai';
import { openai, anthropic } from '@ai-sdk/openai';

const result = await fallback({
  execute: async () => {
    const { text } = await generateText({
      model: openai('gpt-4-turbo'),
      prompt: 'Explain quantum computing'
    });
    return text;
  },
  fallback: async () => {
    const { text} = await generateText({
      model: anthropic('claude-3-5-sonnet-20241022'),
      prompt: 'Explain quantum computing'
    });
    return text;
  }
});

Data Processing Pipeline

import { fanOut } from 'ai-patterns';
import { embed } from 'ai';

const chunks = [
  { id: '1', text: 'Introduction to ML' },
  { id: '2', text: 'Deep learning basics' },
  // ... more chunks
];

const result = await fanOut({
  items: chunks,
  execute: async (chunk) => {
    const { embedding } = await embed({
      model: openai.embedding('text-embedding-3-small'),
      value: chunk.text
    });
    return { id: chunk.id, embedding };
  },
  concurrency: 5
});

Composing Patterns with Middleware

import { compose, retryMiddleware, timeoutMiddleware } from 'ai-patterns/composition';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

// Compose multiple patterns functionally
const robustAI = compose([
  timeoutMiddleware({ duration: 10000 }),
  retryMiddleware({ maxAttempts: 3, backoffStrategy: 'exponential' })
]);

// Use the composed function
const result = await robustAI(
  async (prompt: string) => {
    const { text } = await generateText({
      model: openai('gpt-4-turbo'),
      prompt
    });
    return text;
  },
  'Explain quantum computing'
);

For detailed pattern documentation:

• Compose Pattern →
• Retry Pattern →
• Timeout Pattern →
• Circuit Breaker →
• Rate Limiter →
• Fan-Out →
• Saga →
• Human-in-the-Loop →
• Smart Context Window →
• Reflection Loop →
• Idempotency →
• A/B Testing →
• Cost Tracking →
• Prompt Versioning →
• Response Validation →

Runnable examples:

• View all examples →

Examples

Basic Examples

Each pattern has a simple runnable example:

• retry-simple.ts
• timeout-simple.ts
• circuit-breaker-simple.ts
• rate-limiter-simple.ts
• fan-out-simple.ts
• saga-simple.ts
• human-in-loop-simple.ts
• reflection-loop-simple.ts
• idempotency-simple.ts
• ab-test-simple.ts
• cost-tracking-simple.ts
• prompt-versioning-simple.ts
• response-validation-simple.ts
• context-window-simple.ts

Advanced Examples

• ab-test-with-cost-tracking.ts - Combine A/B testing with cost tracking for ROI optimization
• prompt-versioning-with-monitoring.ts - Prompt versioning with retry logic and analytics
• response-validation-with-retry.ts - Response validation with timeout, retry, and moderation
• context-window-with-summarization.ts - Context window management with AI summarization

Real-World Examples

Coming soon:

• E-commerce - Order processing with saga, retry, and idempotency
• AI Agent - Chatbot with human escalation and circuit breakers
• Microservices - API gateway with rate limiting and retries

Documentation

Pattern Documentation

• Retry Pattern
• Timeout Pattern
• Circuit Breaker Pattern
• Rate Limiter Pattern
• Fan-Out Pattern
• Saga Pattern
• Human-in-the-Loop Pattern
• Smart Context Window Pattern
• Reflection Loop Pattern
• Idempotency Pattern
• A/B Testing Pattern
• Cost Tracking Pattern
• Prompt Versioning Pattern
• Response Validation Pattern

Guides

• Getting Started
• Pattern Composition
• Error Handling
• Best Practices

API Reference

All patterns follow a consistent API design:

const result = await pattern({
  execute: () => yourFunction(),
  // pattern-specific options...
});

See the API Reference for complete details.

Type Safety

Built with TypeScript strict mode for maximum type safety:

// Full type inference with generics
interface User {
  id: string;
  name: string;
  email: string;
}

const result = await retry<User>({
  execute: async () => {
    return await fetchUser();
  }
});

// result.value is typed as User
const user: User = result.value;
console.log(user.email); // ✅ Full autocomplete

Contributing

Contributions are welcome! Please read our Contributing Guide.

License

MIT © Serge KOKOUA

Acknowledgments

Inspired by:

• Vercel AI SDK - Developer experience
• Polly - Resilience patterns
• ts-retry - TypeScript patterns

Built with ❤️ by Serge KOKOUA

Empowering developers to build solid and robust AI applications.