
Product
Introducing Socket Fix for Safe, Automated Dependency Upgrades
Automatically fix and test dependency updates with socket fix—a new CLI tool that turns CVE alerts into safe, automated upgrades.
ai-fallback
Advanced tools
Automatically switch AI SDK model provider when one of the providers has downtime
Automatically switch between AI model providers when one experiences downtime or errors.
AI providers can encounter:
This package ensures reliability by specifying multiple AI models as fallbacks. It automatically switches to the next available model if the primary fails, maintaining application uptime.
npm install ai-fallback
Reset to the primary model after a delay (e.g., 1 minute):
const model = createFallback({
models: [
anthropic('claude-3-haiku-20240307'), // Use Claude as preferred model
openai('gpt-3.5-turbo'),
],
onError: (error, modelId) => {
console.error(`Error with model ${modelId}:`, error)
},
modelResetInterval: 60000, // Reset to first model after 1 minute of the first error
})
import { createFallback } from 'ai-fallback'
import { openai } from '@ai-sdk/openai'
import { anthropic } from '@ai-sdk/anthropic'
const model = createFallback({
models: [anthropic('claude-3-haiku-20240307'), openai('gpt-3.5-turbo')],
})
The retryAfterOutput
option allows retrying with a different model even if some tokens were already streamed. This is useful when you want to restart the generation from scratch if an error occurs mid-stream:
import { createFallback } from 'ai-fallback'
import { openai } from '@ai-sdk/openai'
import { anthropic } from '@ai-sdk/anthropic'
import { streamText } from 'ai'
let fullText = ''
const model = createFallback({
models: [anthropic('claude-3-haiku-20240307'), openai('gpt-3.5-turbo')],
retryAfterOutput: true, // Enable retrying even after partial output
onError: (err) => {
console.error('Error:', err)
// reset the full text because error happened when some tokens were already streamed in
fullText = ''
},
})
const stream = await streamText({
model,
system: 'You are a helpful assistant.',
messages: [{ role: 'user', content: 'Write a long story.' }],
})
for await (const chunk of stream.textStream) {
fullText += chunk
console.log('Current text:', fullText)
}
Generate text with automatic fallback:
const result = await generateText({
model,
system: 'You are a helpful assistant.',
messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
})
Stream text responses:
const stream = await streamText({
model,
system: 'You are a helpful assistant.',
messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
})
for await (const chunk of stream.textStream) {
console.log(chunk)
}
Stream typed objects using Zod
schemas:
import { z } from 'zod'
const stream = await streamObject({
model,
system: 'You are a helpful assistant.',
messages: [
{
role: 'user',
content: 'Give me a person object with name and age properties.',
},
],
schema: z.object({
name: z.string(),
age: z.number(),
}),
})
for await (const chunk of stream.partialObjectStream) {
console.log(chunk)
}
FAQs
Automatically switch AI SDK model provider when one of the providers has downtime
The npm package ai-fallback receives a total of 954 weekly downloads. As such, ai-fallback popularity was classified as not popular.
We found that ai-fallback demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Automatically fix and test dependency updates with socket fix—a new CLI tool that turns CVE alerts into safe, automated upgrades.
Security News
CISA denies CVE funding issues amid backlash over a new CVE foundation formed by board members, raising concerns about transparency and program governance.
Product
We’re excited to announce a powerful new capability in Socket: historical data and enhanced analytics.