
Security News
Deno 2.2 Improves Dependency Management and Expands Node.js Compatibility
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
Lightweight OpenAI API server on top of Llama local models. Suitable for combined use with Microsoft AutoGen.
The llama_cpp_openai
module provides a lightweight implementation of an OpenAI API server on top of
Llama CPP models. This implementation is particularly designed for use with Microsoft AutoGen and includes support for function calls. The project is structured around the llama_cpp_python
module and is aimed at facilitating the integration of AI models in applications using OpenAI clients or API.
The project is organized into several key directories and files:
llama_cpp_openai: Contains the core implementation of the API server.
__init__.py
: Initialization file for the module._api_server.py
: Defines the OpenAPI server, using FastAPI for handling requests._llama_cpp_functions_chat_handler.py
: Implements the llama-2-functionary
chat handler that supports function calling.examples: Provides example scripts demonstrating the usage of the API server.
README.md
: Overview and description of example scripts.autogen_basic.py
: Basic integration of AutoGen with Llama_CPP using the OpenAI API server.autogen_functions.py
: Sets up an AutoGen chatbot with function calls capabilities.basic.py
: Demonstrates the setup and start of an API server using the Llama library.FAQs
Lightweight OpenAI API server on top of Llama local models. Suitable for combined use with Microsoft AutoGen.
We found that llama_cpp_openai demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
Security News
React's CRA deprecation announcement sparked community criticism over framework recommendations, leading to quick updates acknowledging build tools like Vite as valid alternatives.
Security News
Ransomware payment rates hit an all-time low in 2024 as law enforcement crackdowns, stronger defenses, and shifting policies make attacks riskier and less profitable.