granite-io

Granite IO Processing

🚀 A powerful framework for extending IBM Granite model functionality through input/output processing

Introduction

Granite IO Processing is a framework which enables you to transform how a user calls or infers an IBM Granite model and how the output from the model is returned to the user. In other words, the framework allows you to extend the functionality of calling the model.

Getting Started

📋 Requirements

• Python 3.10+

💾 Installation

We recommend using a Python virtual environment with Python 3.10+. Here is how to setup a virtual environment using Python venv:

python3 -m venv granite_io_venv
source granite_io_venv/bin/activate

[!TIP] If you use pyenv, Conda Miniforge or other such tools for Python version management, create the virtual environment with that tool instead of venv. Otherwise, you may have issues with installed packages not being found as they are linked to your Python version management tool and not venv.

There are 2 ways to install the Granite IO Processor as follows:

📦 From Release

To install from release (PyPi package):

python3 -m venv granite_io_venv
source granite_io_venv/bin/activate
pip install granite-io

🔧 From Source

To install from source(GitHub Repository):

python3 -m venv granite_io_venv
source granite_io_venv/bin/activate
git clone https://github.com/ibm-granite/granite-io
cd granite-io
pip install -e .

🎯 Framework Example

Sample code snippet showing how to use the framework:

from granite_io import make_backend, make_io_processor
from granite_io.types import ChatCompletionInputs, UserMessage

model_name = "granite3.2:8b"
io_processor = make_io_processor(
    model_name, backend=make_backend("openai", {"model_name": model_name})
)
messages=[
    UserMessage(
        content="What's the fastest way for a seller to visit all the cities in their region?",
    )
]

# Without Thinking
outputs = io_processor.create_chat_completion(ChatCompletionInputs(messages=messages))
print("------ WITHOUT THINKING ------")
print(outputs.results[0].next_message.content)

# With Thinking
outputs = io_processor.create_chat_completion(
    ChatCompletionInputs(messages=messages, thinking=True)
)
print("------ WITH THINKING ------")
print(">> Thoughts:")
print(outputs.results[0].next_message.reasoning_content)
print(">> Response:")
print(outputs.results[0].next_message.content)

📊 Sample Output

When you run the above example, you should see output similar to this:

$ python test.py
------ WITHOUT THINKING ------
The problem you're describing is a variant of the well-known Traveling Salesman Problem (TSP), which involves finding the shortest possible route that visits a given set of cities and returns to the origin. For a specific region, the optimal solution depends on the geographical layout and the distance between each city pair.

Here are some general strategies for solving this problem:

1. **Exact Algorithm**: Use mathematical optimization techniques like branch-and-bound or dynamic programming to find an exact solution if your dataset isn't too large. These methods can provide the shortest route, but they might be computationally expensive for a large number of cities.

2. **Approximation Algorithms**: For larger datasets, exact algorithms may become impractical due to computational complexity. In such cases, heuristic or approximation algorithms can provide near-optimal solutions more efficiently. Examples include the Nearest Neighbor algorithm, 2-opt swap, Lin–Kernighan–Helsgaun algorithm, and Christofides Algorithm (for metric TSP).

3. **Genetic Algorithms**: These are evolutionary computation techniques inspired by natural selection and genetics. Genetic algorithms can handle larger datasets and may find a near-optimal solution in less time than exact methods, though they do not guarantee optimality.

4. **Machine Learning/AI Approaches**: Machine learning models, such as deep reinforcement learning, have been developed to solve TSP, especially for large instances where traditional methods struggle.

Remember that the optimal method would depend on the number of cities (nodes) and their distribution, computational resources at hand, and the level of accuracy required. 

It's also worth considering additional constraints like time-of-day traffic, road conditions, fuel efficiency, etc., which might influence the route in real-world scenarios. These can be integrated into more advanced algorithms and models for a more practical, efficient solution.
------ WITH THINKING ------
>> Thoughts:
To solve this problem, we need to consider it as a variant of the Traveling Salesman Problem (TSP), which is an NP-hard problem in combinatorial optimization. This means there isn't a straightforward, quick solution for large datasets due to its computational complexity. However, there are several approaches and heuristics that can provide solutions, even if not optimal, in a reasonable amount of time for practical applications.

1. Identify the number of cities (or nodes) - knowing this will help in selecting an appropriate algorithm or method.
2. Consider the constraints: Is there a time limit? Are there specific routes that should be avoided?
3. Decide on the scale and resources available for computation, as some methods are more suited for smaller datasets or require significant computational power.
4. Explore various algorithms designed to solve TSP, like brute force, dynamic programming, nearest neighbor, genetic algorithms, or approximation algorithms.
>> Response:
The problem you're asking about—visiting all cities in a salesman's region exactly once and then returning to the origin—is essentially a variation of the classical Traveling Salesman Problem (TSP). Given its complexity, there isn't an outright "fastest" method that works optimally for every scenario, especially as the number of cities increases. However, several strategies can provide acceptable solutions within practical time constraints:

1. **Greedy Algorithm - Nearest Neighbor:**
   This is one of the simplest heuristics for TSP and can be executed relatively quickly even on large datasets. The salesman starts at a random city and at each step moves to the nearest unvisited city until all cities have been visited. While not guaranteed to find the shortest route, it provides a decent approximation, especially in larger regions where the impact of local suboptimality is lessened.

2. **Christofides Algorithm (for Euclidean planes):**
   If distances between cities form a metric that satisfies the triangle inequality and cities lie in a 2-dimensional Euclidean space, the Christofides algorithm can be used. This algorithm guarantees a solution within 1.5 times the optimal solution but is more computationally intensive than simpler methods.

3. **Genetic Algorithms:**
   These are powerful optimization techniques inspired by evolutionary biology. They work by iteratively improving upon a population of candidate solutions via processes like selection, crossover, and mutation. Genetic algorithms can provide high-quality solutions for TSP but require more computational resources and time compared to simpler heuristics.

4. **Linear Programming Heuristics:**
   Methods such as the one proposed by Lin and Kernighan (called "Karp's algorithm") or more recent linear programming formulations can yield near-optimal solutions, though they typically necessitate specialized software and significant computational power.

5. **Software Tools & APIs:**
   For immediate application needs without wanting to build a custom solution, leveraging existing tools like Google's OR-Tools is advisable. These platforms have robust implementations of various TSP algorithms that can be configured according to specific requirements (like time limits or distance metrics).

6. **Prioritize Nearby Cities Initially:**
   If geographical constraints make certain regions more accessible (e.g., due to better infrastructure), prioritizing visits within those nearby areas first could save time.

When choosing a method, consider factors like the number of cities, available computational resources, required solution quality, and the specific characteristics of your region (like whether distances follow a grid pattern or are more random). For small-scale problems or immediate application needs, simpler methods might suffice. Larger-scale or time-sensitive applications may benefit from more advanced algorithms or dedicated software solutions.

[!IMPORTANT] To get started with the examples, make sure you have followed the Installation steps first. You will need additional packages to be able to run the OpenAI example. They can be installed by running pip install -e "granite-io[openai]". Replace package name granite-io with . if installing from source.

To be able to run the above code snippet, you will need an Ollama server running locally and IBM Granite 3.2 model cached (ollama pull granite3.2:8b).

Try It Out!

To help you get up and running as quickly as possible with the Granite IO Processing framework, check out the following resources which demonstrate further how to use the framework:

• Python script examples:

[!IMPORTANT] To get started with the examples, make sure you have followed the Installation steps first. You will need additional packages to be able to run the examples. They can be installed by running pip install -e "granite-io[openai]" and pip install -e "granite-io[litellm]. Replace package name granite-io with . if installing from source.

You will also need an Ollama server running locally and IBM Granite 3.2 model cached (ollama pull granite3.2:8b).

• 💬 Granite 3.2 chat request
• 🧠 Granite 3.2 chat request with thinking
• 📚 Granite 3.2 RAG
• 🚨 Granite 3.2 RAG and hallucinations
• 🗳️ Granite 3.2 MBRD majority voting
• 🔧 Granite 3.2 custom IO processor
• 🔀 Granite 3.2 separate input and out processors
• ☁️ Using watsonx.ai

• Jupyter notebook tutorials:

[!IMPORTANT] To get started with the examples, make sure you have followed the Installation steps first. You will also need additional packages to be able to run the Jupyter notebook. They can be installed by running pip install -e "granite-io[transformers]" and pip install -e "granite-io[notebook]". Replace package name granite-io with . if installing from source. The notebooks can be then run with following command jupyter notebook <path_to_notebook>.

• 🤔 Granite 3.2 answerability
• 🎯 Granite 3.2 certainity
• 📖 Granite 3.2 citations
• 🚨 Granite 3.2 hallucinations
• 🔄 Granite 3.2 IO APIs
• ✏️ Granite 3.2 query rewrite
• 📚 Granite 3.2 RAG
• 🔍 Granite 3.2 retrieval
• 🗳️ Granite 3.2 simple majority voting

Documentation and Architecture

This project uses MkDocs with the Material theme for comprehensive documentation. To access and work with the documentation:

• 📁 Documentation source: Located in ./mkdocs/docs/
• ⚙️ Configuration: See ./mkdocs/mkdocs.yml
• 🚀 Local development: Follow the instructions in ./mkdocs/README.md

To serve the documentation locally:

cd mkdocs
pip install mkdocs mkdocs-material
mkdocs serve

Click the badge below to view the documentation site:

🤝 Contributing

Click below to view our contributing guide: