litellm

🚅 LiteLLM.js

JavaScript implementation of LiteLLM.

Usage

npm install litellm

import { completion } from 'litellm';
process.env['OPENAI_API_KEY'] = 'your-openai-key';

const response = await completion({
  model: 'gpt-3.5-turbo',
  messages: [{ content: 'Hello, how are you?', role: 'user' }],
});

// or stream the results
const stream = await completion({
  model: "gpt-3.5-turbo",
  messages: [{ content: "Hello, how are you?", role: "user" }],
  stream: true
});

for await (const part of stream) {
  process.stdout.write(part.choices[0]?.delta?.content || "");
}

Features

We aim to support all features that LiteLLM python package supports.

• Standardised completions
• Standardised embeddings
• Standardised input params 🚧 - List is here
• Caching ❌
• Proxy ❌

Supported Providers

Provider	Completion	Streaming	Embedding
openai	✅	✅	✅
cohere	✅	✅	❌
anthropic	✅	✅	❌
ollama	✅	✅	✅
ai21	✅	✅	❌
replicate	✅	✅	❌
deepinfra	✅	✅	❌
mistral	✅	✅	✅
huggingface	❌	❌	❌
together_ai	❌	❌	❌
openrouter	❌	❌	❌
vertex_ai	❌	❌	❌
palm	❌	❌	❌
baseten	❌	❌	❌
azure	❌	❌	❌
sagemaker	❌	❌	❌
bedrock	❌	❌	❌
vllm	❌	❌	❌
nlp_cloud	❌	❌	❌
aleph alpha	❌	❌	❌
petals	❌	❌	❌

Development

Clone the repo

git clone https://github.com/zya/litellmjs.git

Install dependencies

npm install

Run unit tests

npm t

Run E2E tests

First copy the example env file.

cp .example.env .env

Then fill the variables with your API keys to be able to run the E2E tests.

OPENAI_API_KEY=<Your OpenAI API key>
....

Then run the command below to run the tests

npm run test:e2e