gllm-inference-binary

GLLM Inference

Description

A library containing components related to model inferences in Gen AI applications.

Installation

Prerequisites

• Python 3.11+ - Install here
• Pip (if using Pip) - Install here
• Poetry 1.8.1+ (if using Poetry) - Install here
• Git (if using Git) - Install here
• For git installation:
  • Access to the GDP Labs SDK github repository

1. Installation from Artifact Registry

Choose one of the following methods to install the package:

Using pip

pip install gllm-inference-binary

Using Poetry

poetry add gllm-inference-binary

2. Development Installation (Git)

For development purposes, you can install directly from the Git repository:

poetry add "git+ssh://[e-mail]/GDP-ADMIN/gen-ai-internal.git#subdirectory=libs/gllm-inference"

Available extras:

• anthropic: Install Anthropic models dependencies
• google-genai: Install Google Generative AI models dependencies
• google-vertexai: Install Google Vertex AI models dependencies
• huggingface: Install HuggingFace models dependencies
• openai: Install OpenAI models dependencies
• twelvelabs: Install TwelveLabs models dependencies

Managing Dependencies

• Go to root folder of gllm-inference module, e.g. cd libs/gllm-inference.
• Run poetry shell to create a virtual environment.
• Run poetry lock to create a lock file if you haven't done it yet.
• Run poetry install to install the gllm-inference requirements for the first time.
• Run poetry update if you update any dependency module version at pyproject.toml.

Contributing

Please refer to this Python Style Guide to get information about code style, documentation standard, and SCA that you need to use when contributing to this project

• Activate pre-commit hooks using pre-commit install
• Run poetry shell to create a virtual environment.
• Run poetry lock to create a lock file if you haven't done it yet.
• Run poetry install to install the gllm-inference requirements for the first time.
• Run which python to get the path to be referenced at Visual Studio Code interpreter path (Ctrl+Shift+P or Cmd+Shift+P)
• Try running the unit test to see if it's working:

poetry run pytest -s tests/unit_tests/