Security News
Cloudflare Adds Security.txt Setup Wizard
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
@axflow/models
Advanced tools
Zero-dependency, modular SDK for building robust natural language applications
Zero-dependency, modular SDK for building robust natural language applications.
npm i @axflow/models
fetch
and the stream APIsfetch
implementation for request middleware (e.g., custom headers, logging)See the guides at docs.axilla.io.
import {OpenAIChat} from '@axflow/models/openai/chat';
import {CohereGenerate} from '@axflow/models/cohere/generate';
import {StreamToIterable} from '@axflow/models/shared';
const gpt4Stream = OpenAIChat.stream(
{
model: 'gpt-4',
messages: [{ role: 'user', content: 'What is the Eiffel tower?' }],
},
{
apiKey: '<openai api key>',
},
);
const cohereStream = CohereGenerate.stream(
{
model: 'command-nightly',
prompt: 'What is the Eiffel tower?',
},
{
apiKey: '<cohere api key>',
},
);
// StreamToIterable is optional in recent node versions as
// ReadableStreams already implement the async iterator protocol
for await (const chunk of StreamToIterable(gpt4Stream)) {
console.log(chunk.choices[0].delta.content);
}
for await (const chunk of StreamToIterable(cohereStream)) {
console.log(chunk.text);
}
For models that support streaming, there is a convenience method for streaming only the string tokens.
import {OpenAIChat} from '@axflow/models/openai/chat';
const tokenStream = OpenAIChat.streamTokens(
{
model: 'gpt-4',
messages: [{ role: 'user', content: 'What is the Eiffel tower?' }],
},
{
apiKey: '<openai api key>',
},
);
// Example stdout output:
//
// The Eiffel Tower is a renowned wrought-iron landmark located in Paris, France, known globally as a symbol of romance and elegance.
//
for await (const token of tokenStream) {
process.stdout.write(token);
}
process.stdout.write("\n");
useChat
hook for dead simple UI integrationWe've made building chat and completion UIs trivial. It doesn't get any easier than this 🚀
///////////////////
// On the server //
///////////////////
import { OpenAIChat } from '@axflow/models/openai/chat';
import { StreamingJsonResponse, type MessageType } from '@axflow/models/shared';
export const runtime = 'edge';
export async function POST(request: Request) {
const { messages } = await request.json();
const stream = await OpenAIChat.streamTokens(
{
model: 'gpt-4',
messages: messages.map((msg: MessageType) => ({ role: msg.role, content: msg.content })),
},
{
apiKey: process.env.OPENAI_API_KEY!,
},
);
return new StreamingJsonResponse(stream);
}
///////////////////
// On the client //
///////////////////
import { useChat } from '@axflow/models/react';
function ChatComponent() {
const {input, messages, onChange, onSubmit} = useChat();
return (
<>
<Messages messages={messages} />
<Form input={input} onChange={onChange} onSubmit={onSubmit} />
</>
);
}
Sometimes you just want to create a proxy to the underlying LLM API. In this example, the server intercepts the request on the edge, adds the proper API key, and forwards the byte stream back to the client.
Note this pattern works exactly the same with our other models that support streaming, like Cohere and Anthropic.
import { NextRequest, NextResponse } from 'next/server';
import { OpenAIChat } from '@axflow/models/openai/chat';
export const runtime = 'edge'
// POST /api/openai/chat
export async function POST(request: NextRequest) {
const chatRequest = await request.json();
// We'll stream the bytes from OpenAI directly to the client
const stream = await OpenAIChat.streamBytes(chatRequest, {
apiKey: process.env.OPENAI_API_KEY!,
});
return new NextResponse(stream);
}
On the client, we can use OpenAIChat.stream
with a custom apiUrl
in place of the apiKey
that points to our Next.js edge route.
DO NOT expose api keys to your frontend.
import { OpenAIChat } from '@axflow/models/openai/chat';
import { StreamToIterable } from '@axflow/models/shared';
const stream = await OpenAIChat.stream(
{
model: 'gpt-4',
messages: [{ role: 'user', content: 'What is the Eiffel tower?' }],
},
{
apiUrl: "/api/openai/chat",
}
);
for await (const chunk of StreamToIterable(stream)) {
console.log(chunk.choices[0].delta.content);
}
import {OpenAIChat} from '@axflow/models/openai/chat';
import type {OpenAIChatTypes} from '@axflow/models/openai/chat';
OpenAIChat.run(/* args */)
OpenAIChat.stream(/* args */)
OpenAIChat.streamBytes(/* args */)
OpenAIChat.streamTokens(/* args */)
import {OpenAICompletion} from '@axflow/models/openai/completion';
import type {OpenAICompletionTypes} from '@axflow/models/openai/completion';
OpenAICompletion.run(/* args */)
OpenAICompletion.stream(/* args */)
OpenAICompletion.streamBytes(/* args */)
OpenAICompletion.streamTokens(/* args */)
import {OpenAIEmbedding} from '@axflow/models/openai/embedding';
import type {OpenAIEmbeddingTypes} from '@axflow/models/openai/embedding';
OpenAIEmbedding.run(/* args */)
import {CohereGeneration} from '@axflow/models/cohere/generation';
import type {CohereGenerationTypes} from '@axflow/models/cohere/generation';
CohereGeneration.run(/* args */)
CohereGeneration.stream(/* args */)
CohereGeneration.streamBytes(/* args */)
CohereGeneration.streamTokens(/* args */)
import {CohereEmbedding} from '@axflow/models/cohere/embedding';
import type {CohereEmbeddingTypes} from '@axflow/models/cohere/embedding';
CohereEmbedding.run(/* args */)
import {AnthropicCompletion} from '@axflow/models/anthropic/completion';
import type {AnthropicCompletionTypes} from '@axflow/models/anthropic/completion';
AnthropicCompletion.run(/* args */)
AnthropicCompletion.stream(/* args */)
AnthropicCompletion.streamBytes(/* args */)
AnthropicCompletion.streamTokens(/* args */)
import {useChat} from '@axflow/models/react';
import type {UseChatOptionsType, UseChatResultType} from '@axflow/models/shared';
useChat
is a react hook that makes building chat componets a breeze.
import {StreamToIterable, NdJsonStream, StreamingJsonResponse, HttpError, isHttpError} from '@axflow/models/shared';
import type {NdJsonValueType, JSONValueType, MessageType} from '@axflow/models/shared';
FAQs
Zero-dependency, modular SDK for building robust natural language applications
The npm package @axflow/models receives a total of 139 weekly downloads. As such, @axflow/models popularity was classified as not popular.
We found that @axflow/models demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
Security News
The Socket Research team breaks down a malicious npm package targeting the legitimate DOMPurify library. It uses obfuscated code to hide that it is exfiltrating browser and crypto wallet data.
Security News
ENISA’s 2024 report highlights the EU’s top cybersecurity threats, including rising DDoS attacks, ransomware, supply chain vulnerabilities, and weaponized AI.