async-openai
Unofficial Async Python client library for the OpenAI API based on Documented Specs
Latest Version: ![PyPI version](https://badge.fury.io/py/async-openai.svg)
Official Client
Features
Installation
pip install async-openai
pip install git+https://github.com/GrowthEngineAI/async-openai.git
Quick Usage
import asyncio
from async_openai import OpenAI, settings, CompletionResponse
OpenAI.configure(
api_key = "sk-XXXX",
organization = "org-XXXX",
debug_enabled = False,
)
result: CompletionResponse = OpenAI.completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
print(result.text)
print(len(result))
print(result.consumption)
result: CompletionResponse = asyncio.run(
OpenAI.completions.async_create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
)
Configuration and Environment Variables
The following environment variables can be used to configure the client.
OpenAI Configuration
url: The OpenAI API URL | Env: [`OPENAI_API_URL`]
scheme: The OpenAI API Scheme | Env: [`OPENAI_API_SCHEME`]
host: The OpenAI API Host | Env: [`OPENAI_API_HOST`]
port: The OpenAI API Port | Env: [`OPENAI_API_PORT`]
api_base: The OpenAI API Base | Env: [`OPENAI_API_BASE`]
api_key: The OpenAI API Key | Env: [`OPENAI_API_KEY`]
api_path: The OpenAI API Path | Env: [`OPENAI_API_PATH`]
api_type: The OpenAI API Type | Env: [`OPENAI_API_TYPE`]
api_version: The OpenAI API Version | Env: [`OPENAI_API_VERSION`]
api_key_path: The API Key Path | Env: [`OPENAI_API_KEY_PATH`]
organization: Organization | Env: [`OPENAI_ORGANIZATION`]
proxies: The OpenAI Proxies | Env: [`OPENAI_PROXIES`]
timeout: Timeout in Seconds | Env: [`OPENAI_TIMEOUT`]
max_retries: The OpenAI Max Retries | Env: [`OPENAI_MAX_RETRIES`]
ignore_errors: Ignore Errors | Env: [`OPENAI_IGNORE_ERRORS`]
disable_retries: Disable Retries | Env: [`OPENAI_DISABLE_RETRIES`]
max_connections: Max Connections | Env: [`OPENAI_MAX_CONNECTIONS`]
max_keepalive_connections: Max Keepalive Connections | Env: [`OPENAI_MAX_KEEPALIVE_CONNECTIONS`]
keepalive_expiry: Keepalive Expiry | Env: [`OPENAI_KEEPALIVE_EXPIRY`]
custom_headers: Custom Headers | Env: [`OPENAI_CUSTOM_HEADERS`]
Azure Configuration
azure_url: The OpenAI API URL | Env: [`AZURE_OPENAI_API_URL`]
azure_scheme: The OpenAI API Scheme | Env: [`AZURE_OPENAI_API_SCHEME`]
azure_host: The OpenAI API Host | Env: [`AZURE_OPENAI_API_HOST`]
azure_port: The OpenAI API Port | Env: [`AZURE_OPENAI_API_PORT`]
azure_api_key: The OpenAI API Key | Env: [`AZURE_OPENAI_API_KEY`]
azure_api_base: The OpenAI API Base | Env: [`AZURE_OPENAI_API_BASE`]
azure_api_path: The OpenAI API Path | Env: [`AZURE_OPENAI_API_PATH`]
azure_api_type: The OpenAI API Type | Env: [`AZURE_OPENAI_API_TYPE`]
azure_api_version: The OpenAI API Version | Env: [`AZURE_OPENAI_API_VERSION`]
azure_api_key_path: The API Key Path | Env: [`AZURE_OPENAI_API_KEY_PATH`]
azure_organization: Organization | Env: [`AZURE_OPENAI_ORGANIZATION`]
azure_proxies: The OpenAI Proxies | Env: [`AZURE_OPENAI_PROXIES`]
azure_timeout: Timeout in Seconds | Env: [`AZURE_OPENAI_TIMEOUT`]
azure_max_retries: The OpenAI Max Retries | Env: [`AZURE_OPENAI_MAX_RETRIES`]
Initialize Clients Manually, and working with multiple clients
from async_openai import OpenAI
OpenAI.configure(
api_key = "sk-XXXX",
organization = "org-XXXX",
debug_enabled = False,
azure_api_base = 'https://....openai.azure.com/',
azure_api_version = '2023-07-01-preview',
azure_api_key = '....',
)
oai = OpenAI.init_api_client()
az = OpenAI.init_api_client('az', set_as_default = False, debug_enabled = True)
result = az.completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
result = OpenAI.completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
result = oai.completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
result = OpenAI['az'].completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
result = OpenAI['default'].completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
result = OpenAI[0].completions.create(
prompt = 'say this is a test',
max_tokens = 4,
stream = True
)
Handling Errors, Retries, and Rotations
The below will show you how to rotate between multiple clients when you hit an error.
Important Auto-rotation is only supported with chat_create
and async_chat_create
methods. Otherwise, you should handle the rotation manually.
import asyncio
from async_openai import OpenAI, ChatResponse
from async_openai.utils import logger
OpenAI.configure(
api_key = "sk-XXXX",
organization = "org-XXXX",
debug_enabled = False,
azure_api_base = 'https://....openai.azure.com/',
azure_api_version = '2023-07-01-preview',
azure_api_key = '....',
enable_rotating_clients = True,
prioritize = "azure",
)
OpenAI.get_current_client_info(verbose = True)
result: ChatResponse = OpenAI.chat_create(
model = "gpt-3.5-turbo-16k",
messages = [
{"role": "user", "content": "Translate the following English text to French: “Multiple models, each with different capabilities and price points. Prices are per 1,000 tokens. You can think of tokens as pieces of words, where 1,000 tokens is about 750 words. This paragraph is 35 tokens”"}
],
auto_retry = True,
)
logger.info(f'Result Chat Message: {result.messages}')
logger.info(f'Result Usage: {result.usage}')
logger.info(f'Result Consumption: {result.consumption}')
result: ChatResponse = asyncio.run(
OpenAI.async_chat_create(
model = "gpt-3.5-turbo-16k",
messages = [
{"role": "user", "content": "Translate the following English text to French: “Multiple models, each with different capabilities and price points. Prices are per 1,000 tokens. You can think of tokens as pieces of words, where 1,000 tokens is about 750 words. This paragraph is 35 tokens”"}
],
auto_retry = True,
)
)
Function Calls
The latest version of the API allows for function calls to be made. This is currently only supported in Chat
and requires api version: 2023-07-01-preview
for azure
.
Function calls support using pydantic
models to auto-generate the schemas
import asyncio
from enum import Enum
from client_rotate import OpenAI
from async_openai.utils import logger
from pydantic import BaseModel, Field
class Unit(str, Enum):
celsius = "celsius"
fahrenheit = "fahrenheit"
class Weather(BaseModel):
location: str = Field(..., description="The city and state, e.g. San Francisco, CA.")
unit: Unit = Field(Unit.fahrenheit)
functions = [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": Weather,
}
]
result: ChatResponse = OpenAI.chat_create(
model = "gpt-3.5-turbo-16k",
messages = [
{"role": "user", "content": "What's the weather like in Boston today?"}
],
functions = functions,
auto_retry = True,
)
logger.info(f'Result Chat Message: {result.messages}')
logger.info(f'Result Chat Function: {result.function_results}')
logger.info(f'Result Usage: {result.usage}')
logger.info(f'Result Consumption: {result.consumption}')
"""
Result:
> Result Chat Message: [ChatMessage(content='', role='assistant', function_call=FunctionCall(name='get_current_weather', arguments={'location': 'Boston, MA'}), name=None)]
> Result Chat Function: [FunctionCall(name='get_current_weather', arguments={'location': 'Boston, MA'})]
> Result Usage: prompt_tokens=16 completion_tokens=19 total_tokens=35
> Result Consumption: 0.00012399999999999998
"""
Configure Azure Model Mapping
Your azure models may be named differently than the default mapping. By configuring the mapping, you can automatically map the models to the correct azure model (when using openai model names).
from async_openai import OpenAI
"""
Default Azure Model Mapping
{
'gpt-3.5-turbo': 'gpt-35-turbo',
'gpt-3.5-turbo-16k': 'gpt-35-turbo-16k',
'gpt-3.5-turbo-instruct': 'gpt-35-turbo-instruct',
'gpt-3.5-turbo-0301': 'gpt-35-turbo-0301',
'gpt-3.5-turbo-0613': 'gpt-35-turbo-0613',
}
"""
AzureModelMapping = {
'gpt-3.5-turbo': 'azure-gpt-35-turbo',
'gpt-3.5-turbo-16k': 'azure-gpt-35-turbo-16k',
'gpt-3.5-turbo-instruct': 'azure-gpt-35-turbo-instruct',
'gpt-3.5-turbo-0301': 'azure-gpt-35-turbo-0301',
'gpt-3.5-turbo-0613': 'azure-gpt-35-turbo-0613',
}
OpenAI.configure(
api_key = "sk-XXXX",
organization = "org-XXXX",
debug_enabled = False,
azure_api_base = 'https://....openai.azure.com/',
azure_api_version = '2023-07-01-preview',
azure_api_key = '....',
azure_model_mapping = AzureModelMapping,
)
OpenAI.init_api_client('az', set_as_default = True, debug_enabled = True)
result: ChatResponse = OpenAI.chat.create(
model = "gpt-3.5-turbo-16k",
messages = [
{"role": "user", "content": "Translate the following English text to French: “Multiple models, each with different capabilities and price points. Prices are per 1,000 tokens. You can think of tokens as pieces of words, where 1,000 tokens is about 750 words. This paragraph is 35 tokens”"}
],
auto_retry = True,
)
Dependencies
The aim of this library is to be as lightweight as possible. It is built on top of the following libraries:
-
aiohttpx: Unified Async / Sync HTTP Client that wraps around httpx
-
pydantic: Type Support
-
file-io: Async Cloud-based File Storage I/O
-
backoff: Retries with Exponential Backoff