agentsiq
Advanced tools
The Ultimate LLM Selection Engine — AgentsIQ automatically chooses the most cost-efficient, fastest, and highest-quality model for each task, supporting 10+ models including OpenAI, Anthropic, Google, Ollama (local), and Grok. Features comprehensive benchmarking with beautiful visualizations and real-time performance analytics.
https://github.com/yourusername/AgentsIQ/assets/youruserid/AgentIQ.mp4
See AgentsIQ intelligently route between models in real-time
/control without redeploying.RouterManager ensures both the runner and dashboard operate on the same model router instance in a single process (examples/serve_and_demo.py).config.yaml — no code edits required.pip install agentsiq
git clone https://github.com/yourusername/AgentsIQ.git
cd AgentsIQ
python -m venv venv && source venv/bin/activate # Windows: venv\Scripts\activate
pip install -e .
git clone https://github.com/yourusername/AgentsIQ.git
cd AgentsIQ
python -m venv venv && source venv/bin/activate
pip install -e ".[dev,docs]"
Create a .env file with your API keys:
# Required for cloud models
OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key
GOOGLE_API_KEY=your_google_key
GROK_API_KEY=your_grok_key
# Optional: Ollama configuration (defaults to localhost:11434)
OLLAMA_URL=http://localhost:11434
# Optional: AgentOps for advanced analytics
AGENTOPS_API_KEY=your_agentops_key
python examples/benchmark.py
Choose option 1 for the comprehensive model comparison with beautiful visualizations!
# Start Jupyter Lab
jupyter lab examples/notebooks/
# Or start Jupyter Notebook
jupyter notebook examples/notebooks/
Available Notebooks:
See Notebook Examples for detailed learning paths and tutorials.
| Provider | Models | Cost/1K tokens | Best For | Logo |
|---|---|---|---|---|
 | GPT-4o, GPT-4o-mini | $5.00, $0.15 | Code generation, complex reasoning | 🤖 |
 | Claude-3-Haiku | $0.25 | Summarization, analysis | 🧠 |
 | Gemini-Pro | $0.125 | General purpose, fast responses | 🔍 |
 | Llama3.1 (8B/70B), Qwen2.5 (7B/72B) | $0.00 | Local inference, privacy | 🏠 |
 | Grok-2, Grok-2-Vision | $0.20-0.25 | Creative writing, humor | 😄 |
from agentsiq.router import ModelRouter
router = ModelRouter()
model = router.select_model("Write a Python function to sort a list")
response, quality = router.call_model(model, "Write a Python function to sort a list")
print(f"Selected: {model}, Quality: {quality}")
📖 Want to understand how the intelligent selection works?
Check out our Architecture Documentation for a detailed explanation of the multi-objective optimization algorithm and decision-making process.
from agentsiq.agent import Agent
from agentsiq.collab import Collab
from agentsiq.router import ModelRouter
researcher = Agent("Researcher", "Finds information", "openai:gpt-4o-mini", ["retrieval"])
analyst = Agent("Analyst", "Summarizes info", "anthropic:claude-3-haiku", ["summarize"])
router = ModelRouter()
collab = Collab([researcher, analyst], {"retrieval": lambda _: "[retrieval] ok", "summarize": lambda _: "[summary] ok"})
result = collab.run("Analyze the latest trends in AI")
python -m examples.serve_and_demo
# open http://127.0.0.1:8000 → Summary, Decisions, Control, Health
The enhanced benchmark system provides comprehensive analysis across multiple dimensions:
🏆 MODEL PERFORMANCE SUMMARY
------------------------------
Model Avg Response Time (s) Avg Cost ($) Avg Quality Score Tasks Completed
ollama:qwen2.5:7b 0.2500 0.0000 0.8000 5
ollama:llama3.1:8b 0.3000 0.0000 0.8200 5
google:gemini-pro 0.6000 0.1250 0.7800 5
anthropic:claude-3-haiku 0.7000 0.2500 0.8000 5
openai:gpt-4o-mini 0.8000 0.1500 0.7500 5
grok:grok-2 0.9000 0.2000 0.9000 5
openai:gpt-4o 1.0000 5.0000 0.9500 5
Comprehensive 9-chart analysis showing model performance across all dimensions
Chart Breakdown:
# macOS/Linux
curl -fsSL https://ollama.ai/install.sh | sh
# Windows
# Download from https://ollama.ai/download
# Fast models (recommended for testing)
ollama pull llama3.1:8b
ollama pull qwen2.5:7b
# High-quality models (requires more RAM)
ollama pull llama3.1:70b
ollama pull qwen2.5:72b
ollama list
# Should show your downloaded models
python setup_ollama.py
This script will:
graph TB
subgraph "User Input"
A[Task Request] --> B[Task Analysis]
end
subgraph "Intelligent Router"
B --> C[Traits Detection]
C --> D[Token Estimation]
D --> E[Model Scoring]
E --> F[Multi-Objective Optimization]
F --> G[Model Selection]
end
subgraph "Model Providers"
H[OpenAI<br/>GPT-4o, GPT-4o-mini]
I[Anthropic<br/>Claude-3-Haiku]
J[Google<br/>Gemini-Pro]
K[Ollama<br/>Llama3.1, Qwen2.5]
L[Grok<br/>Grok-2, Grok-2-Vision]
end
subgraph "Scoring Engine"
M[Cost Analysis<br/>Weight: 60%]
N[Latency Analysis<br/>Weight: 25%]
O[Quality Analysis<br/>Weight: 15%]
P[Task-Specific Boosts]
end
subgraph "Output & Analytics"
Q[Response Generation]
R[Quality Metrics]
S[Decision Logging]
T[AgentOps Analytics]
end
G --> H
G --> I
G --> J
G --> K
G --> L
E --> M
E --> N
E --> O
E --> P
H --> Q
I --> Q
J --> Q
K --> Q
L --> Q
Q --> R
Q --> S
Q --> T
style A fill:#e1f5fe
style G fill:#c8e6c9
style Q fill:#fff3e0
style T fill:#f3e5f5
AgentsIQ's intelligent multi-objective optimization system
📖 Detailed Architecture: See Architecture Documentation for complete technical details.
AgentsIQ seamlessly integrates with AgentOps for advanced analytics and observability:
# Add to your .env file
AGENTOPS_API_KEY=your_agentops_key_here
smart | cheapest | fastest | hybridagentsiq/router.py — SMART routing + cost/latency/quality scoringagentsiq/router_manager.py — singleton router (shared in‑process)agentsiq/dashboard.py — dark HTML UI, /control, /summary, /decisionsagentsiq/agentops_metrics.py — metrics CSV + percentilesagentsiq/decision_store.py — decision rationale JSONexamples/serve_and_demo.py — single‑process server + demo runnerOpen docs/metrics_demo.html locally in your browser — it renders this data with inline SVG sparklines and a dark table:
{
"Researcher": {
"calls": 7,
"avg_latency": 7.43354994910104,
"avg_confidence": 0.781428571428572,
"models": {"openai:gpt-4o-mini": 3, "google:gemini-pro": 4},
"series": [14.05, 21.05, 10.26, 1.74, 2.06, 1.18, 1.69],
"p50": 2.0557, "p90": 16.8537, "p99": 20.6342
},
"Analyst": {
"calls": 7,
"avg_latency": 0.2281,
"avg_confidence": 0.78,
"models": {"tool:summarize": 2, "anthropic:claude-3-haiku": 1, "google:gemini-pro": 4},
"series": [0, 0, 1.2955, 0.0757, 0.0778, 0.0690, 0.0789],
"p50": 0.0757, "p90": 0.5655, "p99": 1.2225
}
}
If you see UnicodeEncodeError from AgentOps (emoji in logs), this project applies a SafeConsoleFilter automatically when AGENTOPS_API_KEY is set. You can also set PYTHONUTF8=1 or PYTHONIOENCODING=utf-8 to allow emojis.
We welcome contributions! Please see our Contributing Guide for details.
This project is licensed under the MIT License - see the LICENSE file for details.
Made with ❤️ by the AgentsIQ Team
⭐ Star this repo if you find it helpful!
FAQs
Intelligent Multi-Model Router for LLM Selection
We found that agentsiq demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
/Product
Socket Firewall Free is now bundled into Docker Hardened Images, adding build-time and dependency-install supply chain protection on top of hardened base images for Node.js, Python, and Rust.
Security News
Season’s greetings from Socket, and here’s to a calm end of year: clean dependencies, boring pipelines, no surprises.
Research
/Security News
Impostor NuGet package Tracer.Fody.NLog typosquats Tracer.Fody and its author, using homoglyph tricks, and exfiltrates Stratis wallet JSON/passwords to a Russian IP address.
