volante
Advanced tools
A transparent, user-owned model router and orchestration control plane (alpha). A supervisor model decomposes a goal into a task DAG, routes each sub-task across the user's explicitly configured models, runs them one-shot or in an agentic tool loop, and synthesizes a final answer. Hard capabilities are enforced first; explainable metadata and quality evidence rank the eligible models. Volante runs locally and keeps inventory, policy, credentials, and decision traces under the user's control.
Selection quality is an evidence-based prediction, not a claim of a universal winner. Volante does not yet ship published representative cross-provider benchmarks or automatic score calibration. Built without an orchestration framework (no LangChain / CrewAI / LiteLLM).
A volante steers the game: the deep-lying midfielder who reads the whole pitch and sends the ball where it does the most good — then takes it back. One mind, many players.
quality, local, cheap, and cash_protect_quota are distinct
routing objectives.AnthropicProvider and a generic
OpenAICompatProvider speak to Anthropic, Google AI Studio (Gemini), Groq, OpenRouter, DeepSeek,
Moonshot (Kimi), local Ollama, and any other OpenAI-compatible endpoint. The inventory and
credentials stay in the user's environment.run_python
in a Docker-isolated sandbox when a daemon is available, withheld when none is — plus host-mediated
fetch_url / read_file).CostMeter tallies per-model usage and cost, and propagates an estimated
flag when a provider returns no usage.FakeProvider + local subprocesses), ruff-clean.flowchart TD
goal(["goal"]) --> S["Supervisor<br/>plan → validated task DAG<br/>(acyclic · typed · one_shot | agentic)"]
S --> R["Router<br/>best predicted fit per task<br/>(hard constraints → explainable score)"]
R --> P
subgraph wave["wave execution · asyncio fan-out · fail-fast"]
direction TB
P["Projector<br/>scoped, budget-capped request<br/>(system + task + deps)"]
P --> W["Worker<br/>one-shot"]
P --> AW["AgenticWorker<br/>model ↔ tool loop<br/>(run_python · fetch_url · read_file)"]
end
W --> BB[("Blackboard<br/>append-only · provenance · latest-wins")]
AW --> BB
BB --> SY["Synthesizer<br/>combine artifacts → final answer"]
SY --> result(["result<br/>+ CostMeter totals · usage · duration"])
classDef io stroke:#8b5cf6,stroke-width:2px;
classDef store stroke:#f59e0b,stroke-width:2px;
class goal,result io;
class BB store;
┌──────────────┐
goal ──────────► │ Supervisor │ plan → validated task DAG (acyclic, typed, one_shot|agentic)
└──────┬───────┘
▼
┌──────────────┐ per task: hard-capability filter, then rank the
│ Router │ predicted fit from configured evidence
└──────┬───────┘
▼
┌───────────── wave execution (asyncio, fan-out cap, fail-fast) ─────────────┐
│ ┌───────────┐ scoped, budget-capped request (system + task + deps) │
│ │ Projector │──────────────────────────────────────────────────────────► │
│ └───────────┘ │
│ ▼ ▼ │
│ ┌─────────┐ one-shot ┌───────────────┐ model↔tool loop │
│ │ Worker │ │ AgenticWorker │ (run_python sandbox, │
│ └────┬────┘ └───────┬───────┘ fetch_url, read_file) │
│ └──────────────┬──────────────────┘ │
└───────────────────────┼───────────────────────────────────────────────────┘
▼
┌──────────────────────┐ append-only, provenance, latest-wins
│ Blackboard │◄──────────────────────────────────────
└──────────┬───────────┘
▼
┌──────────────┐
│ Synthesizer │ combine artifacts → final answer
└──────┬───────┘
▼
result (+ CostMeter totals, usage, duration)
| Component | File | Responsibility |
|---|---|---|
| Supervisor | src/volante/supervisor.py | Decompose goal → validated task DAG |
| Router | src/volante/router.py | Whole inventory → eligible candidates → explainable ranking |
| Projector | src/volante/projector.py | Scoped, budget-capped request from blackboard artifacts |
| Worker | src/volante/worker.py | One-shot model call |
| AgenticWorker | src/volante/agent.py | Model↔tool loop with per-turn records |
| Blackboard | src/volante/blackboard.py | Append-only shared state with provenance |
| Synthesizer | src/volante/synthesizer.py | Artifacts → final answer |
| Runtime | src/volante/runtime.py | Orchestrate: plan → waves → synthesize (streaming, fail-fast) |
| Providers | src/volante/providers/ | Anthropic + OpenAI-compatible adapters (complete/stream/tools) |
| Tools | src/volante/tools/ | Sandbox / DockerSandbox, run_python, fetch_url, read_file |
| Eval | eval/ | 7 goals (5 katas + 2 multi-part), 3-arm comparison, forgery-resistant scorer |
Requires Python 3.11.10+ (the floor is a security boundary — see Providers).
Install it:
pip install volante # the CLI and the library
pip install "volante[mcp]" # ...plus the MCP server, for IDE use
volante --version
volante --list-models # what your environment currently offers, offline
volante needs at least one configured provider before it can run a goal — see
Providers for the two-line setup. Until then --list-models and --help work and
everything else will tell you what is missing.
Or work on it: the clone gives you the tests, the Web UI, and a zero-key demo that the wheel deliberately does not ship.
git clone https://github.com/ribato22/volante
cd volante
uv sync --dev # install deps + dev tools
uv run pytest # 800+ tests, no network
uv run ruff check . # lint
# See it orchestrate end-to-end with ZERO API keys (FakeProvider demo):
uv run python examples/fake_provider.py
Then configure at least one real provider (see Providers) and run a demo:
cp .env.example .env # fill in one provider, then `set -a; . .env; set +a`
uv run python demo.py # show detected providers
uv run python demo.py orchestrate # full supervisor → workers → synth, streamed live
uv run python demo.py agentic # one cross-provider agentic coding task (run_python loop)
uv run python demo.py eval # 3-arm eval suite
demo.py orchestrate streams every phase live, then prints the result (illustrative):
Orchestrate demo — planner/synth model=openai/gpt-4o-mini
(planning + workers + synthesis stream live)
[haiku] Threads run as one— / tasks bloom in parallel time, / the join gathers all.
STATUS: success
FINAL:
Threads run as one—
tasks bloom in parallel time,
the join gathers all.
cost: $0.001834
demo.py eval prints the 3-arm table (format_report). Here is what it actually printed on the
last full run:
GOAL WINNER BASE ORCH AGEN
--------------------------------------------------
slugify baseline 1.00 1.00 0.67
roman baseline 1.00 1.00 0.70
calc orchestration 0.96 1.00 0.27
csv_stats baseline 1.00 1.00 0.57
json_flatten baseline 1.00 1.00 0.70
textkit baseline 1.00 1.00 0.47
ledger baseline 1.00 1.00 0.80
toolbelt baseline 1.00 1.00 0.70
debug_gauntlet baseline 0.98 0.98 0.63
--------------------------------------------------
wins: baseline=8 orchestration=1 agentic=0 ties=0
totals: baseline $0.018108 orchestration $0.139354 agentic $0.099910
VERDICT: BASELINE
openai/gpt-4o-mini, 9 goals x 3 arms x k=3 = 81 real runs against a Docker sandbox, read from
results-0.4.2-corrected.json — the self-describing artifact each run writes (models, k, per-goal
scores, costs).
"VERDICT: BASELINE" is a cost tie-break, not a quality result. Orchestration matches baseline on eight goals and beats it on one; baseline takes those eight because it reached the same score for 7.7x less money. On quality the suite is a tie at the ceiling: 0.993 baseline vs 0.997 orchestration.
Earlier releases published orchestration at 0.919 here, losing four goals outright. That was
substantially a grading bug of ours, fixed in cd94390: the scorer closed a fenced code block
at the first ``` even when that fell inside a string literal, so a model that embedded the
requested README inside its own module scored 0.000 with working code in it. It punished
single-self-contained-block answers — the shape a synthesis pass emits — and therefore hit the
orchestration arm far harder than the baseline arm. Every orchestration number published before
that commit is a lower bound.
Reproduce it — this needs your keys and spends real money:
uv run python -m eval.run --k 5 --json results.json
Read that table with its limits, because they are large. Both arms now sit at the ceiling on this suite — baseline 0.993, orchestration 0.997 — so it can no longer discriminate between them at all. A tie at 1.00 is not evidence of equivalence; it is evidence the goals are too easy to measure anything. What these nine goals establish is "orchestration does not lose on tasks one model already solves in a single turn", which is worth knowing and is not the claim the idea rests on.
The claim needs a goal with headroom. resolve (in eval/tasks_depth.py, run with
--suite depth) is one: implement a single function governed by seven overlapping precedence
rules, graded on 48 cases balanced twelve-per-answer so a constant answer scores exactly 0.25.
Same model, temperature 0, n=8:
| arm | runs | mean | stdev | unparsable output |
|---|---|---|---|---|
| baseline | 16 | 0.453 | 0.130 | — |
| orchestration | 8 | 0.742 | 0.212 | 0/8 |
Baseline is pooled over two batches run hours apart. A single batch of 8 gave mean 0.414 with stdev 0.007, and quoting that alone would overstate how consistent it is: a second batch contained one run at 0.938. The pooled figure is the honest one, and the shape it hides is worth stating — baseline scores below 0.50 on 15 of 16 runs, so it is not noisy so much as reliably wrong with one lucky exception.
That is +0.289, Welch t=3.54 (df~10), p<0.005 — the first result on this project where orchestration beats a strong single model on quality rather than tying it. Note how it was won: not by better answers, but by removing failures. Before the reliability work the same arm scored 0.620 with stdev 0.322 and returned a module that does not parse in 2 runs of 8; the mean moved 0.12 and the standard deviation moved 0.11, and it was the second number that bought the significance.
Everything above is one model on one class of small coding tasks, and orchestration costs 7.7x
baseline. --prefer cheap now buys that back where it is not earning anything: it asks the planner
to split only when splitting helps. Measured, n=5 per arm —
| goal | baseline | --prefer quality | --prefer cheap |
|---|---|---|---|
slugify (baseline already perfect) | 1.000 · $0.00046 | 1.000 · 10.6x | 1.000 · 4.7x |
resolve (headroom) | 0.708 · $0.00093 | 0.721 · 4.8x | 0.525 · 4.6x |
On a goal one model already aces it halves the bill for identical output. On the goal with headroom it costs 0.20 of score and saves nothing — which is the trade the flag is for, and why it is opt-in rather than the default.
It is opt-in for a measured reason: choosing automatically does not work. Three signals were tried
and all three failed. The planner's own difficulty labels give resolve the same profile
(medium:2, easy:1) as csv_stats, where baseline already scores 1.00. Told it may return a single
task for a simple goal, it never abstains. Shown its own answer and asked to check it against the
goal, it replies OK on work scoring 0.417. There is no signal at this tier, so the choice belongs
to whoever knows what the task is worth. Whether it pays off for a stronger model, a larger task, or work that genuinely exceeds
one context window is unmeasured, and this project does not claim it. The agentic arm still
failed 2 of its 27 runs.
The benchmark's most useful output so far was not the score. It found three real engine bugs — a deterministic livelock in the agentic loop, an eval arm that failed a model for answering correctly without calling a tool, and a stall guard that gave up after one warning when a second one recovers the run — and fixing those took agentic terminal failures from 79% to 7%. It also refuted two of the author's own hypotheses (that a wider goal would break the ceiling effect; that raising the iteration cap would help). That is what the suite is for.
Volante ships three surfaces: a one-command CLI (the primary entrypoint), an optional Web UI, and an importable library. All three need at least one configured provider — see Providers — or (Web UI only) fall back to a no-key demo.
uv run volante "write a haiku about concurrency, then explain the metaphor"
volante streams the plan, each parallel worker's output (labelled per task), and the synthesis live,
then prints a summary. Flags (volante --help):
| Flag | Description |
|---|---|
--prefer {quality,cash_protect_quota,local,cheap} | routing objective. quality (default) ranks predicted task fit; cash_protect_quota reserves subscription quota; local prioritizes eligible local models; cheap prioritizes configured economic cost |
--provider NAME / -P NAME | restrict the planner/synth baseline to this provider |
--model ID | override the planner/synth model_id |
--list-models | print the complete configured/detected routing inventory and exit; combine with --json for machine-readable output |
--usage | print the recent usage ledger (~/.volante/usage.jsonl) — including runs delegated from an IDE via MCP — and exit; combine with --json |
--json | print the run summary as one parseable JSON line; disables streaming |
--no-stream | disable live streaming of plan/worker/synth text |
--version | print the installed version and exit |
Exit codes: 0 success, 1 run failure, 2 config error (e.g. no provider configured), 130
Ctrl-C (prints whatever partial output had streamed so far — never a raw traceback).
The summary reports billed_usd (real cash spent) vs. credit_usd (subscription/plan
API-equivalent value, not cash), physical subscription_calls, and a per-task route trace. JSON
mode also includes every eligible/rejected candidate, score component, selected model, and model
actually executed after any fallback.
uv sync --extra ui
uv run python -m webui # then open http://127.0.0.1:8000
A small FastAPI + Server-Sent-Events app streams a run live in the browser — the plan, each parallel
worker's output (labelled per task), the synthesis, and the final result with cost and model routes.
It runs with your configured providers, or a built-in FakeProvider demo if none are set (no API
key needed). This is a source-checkout feature — webui/ is not shipped in the built wheel/PyPI
package.
VOLANTE_UI_HOST / VOLANTE_UI_PORT override the bind address. Volante refuses a non-loopback host
unless VOLANTE_UI_AUTH_TOKEN is set, and refuses to carry that token over plain HTTP: set
VOLANTE_UI_TLS_CERT + VOLANTE_UI_TLS_KEY to serve TLS directly, or VOLANTE_UI_TRUST_PROXY=1
if a TLS-terminating reverse proxy sits in front. For remote access also set
VOLANTE_UI_ALLOWED_HOSTS to the comma-separated hostnames clients will use. VOLANTE_UI_MAX_GOAL_CHARS (default 20000) and
VOLANTE_UI_MAX_CONCURRENT_RUNS (default 2) bound input and concurrency. The page inserts all
model output via textContent only (never raw HTML), so streamed text cannot inject markup. A
/usage page (linked from the header, gated by the same VOLANTE_UI_AUTH_TOKEN) shows the usage
ledger below.
Every run from the CLI, the Web UI, and the MCP server best-effort appends one JSON line to a
usage ledger — ~/.volante/usage.jsonl by default (VOLANTE_USAGE_LOG to relocate, empty to disable) —
recording status, cash vs. plan credit, physical subscription calls, duration, the models used, and
a truncated goal. This is how you monitor Volante when an IDE agent delegates goals to it over MCP:
uv run volante --usage # recent runs + totals, newest first
uv run volante --usage --json # the same as one JSON object
The Web UI /usage dashboard reads the same ledger (summary tiles + a recent-runs table). Set
VOLANTE_LOG=debug (or info/warning/error) to raise engine diagnostics on stderr — for an MCP
server that surfaces in the client's server-output pane (e.g. VS Code's Output → the Volante MCP
server); it is silent by default and never writes to stdout.
import asyncio
import volante
async def main() -> None:
registry, providers, model_id = volante.build_providers_from_env()
factory = await volante.make_verified_runtime_factory(
registry, providers, model_id
)
runtime = factory()
result = await runtime.aexecute("your goal")
print(result.status, result.billed_usd, result.credit_usd, result.cost_estimated)
asyncio.run(main())
A Runtime is single-use: it runs one goal, and its accounting, route trace, and planner are
all per-run. Call factory() again for each goal (a second or overlapping aexecute on the same
instance is refused with a RuntimeError rather than silently mixing two runs' numbers). Read
result.cost_estimated alongside the amounts — it is True when a provider reported no token
counts for at least one call, so part of the figures is inferred from configured rates rather than
reported by the provider.
The top-level volante package re-exports the common library API (Runtime, Registry,
Router, RoutingDecision, ModelQualityProfile, inventory helpers,
build_providers_from_env, make_verified_runtime_factory, RunResult, ModelInfo, Task,
LLMProvider, ProviderError — see volante.__all__) so you don't need to reach into submodules.
See examples/ for runnable scripts — including
examples/fake_provider.py, which needs no API key at all. For a
guided tour with hardcoded goals, see the demo script:
uv run python demo.py orchestrate|agentic|eval (walked through in Quickstart).
Volante can drive the official headless CLIs you're already logged into instead of (or alongside) a card-billed API key:
export CLAUDE_CODE_ENABLED=1 # needs `claude` installed and logged in
# CLAUDE_CODE_SYSTEM_PROMPT_MODE=replace is the default — makes `claude -p` behave as a
# raw completion; `append` breaks strict-JSON planning, so leave it unset unless you know why.
export CODEX_ENABLED=1 CODEX_TIER=3 # needs `codex login`
uv run volante "your goal"
⚠️ Subscription runs are cash-free but consume your interactive Claude Code / Codex quota — the same pool your interactive coding sessions draw from. A heavy orchestration run can trip a rate-limit pause. The default
qualityobjective favors the highest predicted fit per task, which can lean on subscription models. Pass--prefer cash_protect_quotato mitigate this — it sends bulk/easy work to cheaper local/free-tier models and reserves subscription models for hard tasks only. A card-billed, free-tier, or local model as planner is recommended: subscription CLIs ignoretemperature, so Volante retries planning with self-correction and can gateclaude -pas planner behind a live parse-plan check (it only plans if it demonstrably emits valid plan JSON).This drives the official headless CLIs (
claude -p,codex exec) that you are already logged into — never the claude.ai / ChatGPT web apps. Scraping those web apps is not implemented (it would violate their Terms of Service).
Volante is a CLI first, so it already works in any editor's integrated terminal (uv run volante "…"). For VSCode there are two extra conveniences:
1. One-keystroke tasks. The repo ships .vscode/tasks.json. Open
Terminal → Run Task… (or press ⌘/Ctrl+Shift+B) and pick:
| Task | What it does |
|---|---|
| Volante: Run goal | Prompts for a goal and orchestrates it (streams plan → workers → synthesis). |
| Volante: Web UI | Serves the live Web UI at http://127.0.0.1:8000. |
| Volante: MCP server (stdio) | Runs the MCP server for AI-agent integration (below). |
| Volante: Test / Lint | pytest / ruff over the project. |
2. MCP server — let the AI inside your editor call Volante. Volante ships an
MCP server (volante_mcp/) exposing one tool,
volante_run(goal, prefer?), that plans → routes → runs → synthesizes and returns the final answer
plus an honest cash/plan-credit footer. Any MCP-capable assistant (Claude Code, Cursor, VS Code
Copilot agent mode, Windsurf) can then delegate whole goals to Volante.
Install it clone-free (recommended), or from a source checkout:
# clone-free — uv fetches the published package + the `mcp` extra on demand:
uvx --from "volante[mcp]==0.5.0" volante-mcp
# or install it and run the console script:
pip install "volante[mcp]==0.5.0" # then:
volante-mcp
# or from a source checkout:
uv sync --extra mcp && uv run --extra mcp python -m volante_mcp
Register it with your client. Claude Code — one command:
claude mcp add volante -- uvx --from "volante[mcp]==0.5.0" volante-mcp
Cursor / VS Code / Windsurf — add to the client's MCP config (e.g. .cursor/mcp.json, or
VS Code's .vscode/mcp.json under a "servers" key):
{
"mcpServers": {
"volante": {
"command": "uvx",
"args": ["--from", "volante[mcp]==0.5.0", "volante-mcp"]
}
}
}
The server reads providers from the environment exactly like the CLI (including
CLAUDE_CODE_ENABLED / CODEX_ENABLED), so configure at least one provider first — it does not
fall back to a demo. A full branded VSCode extension is intentionally not shipped; the CLI,
tasks, and the MCP server cover the same ground.
In the official MCP registry. Volante is published to the
official MCP registry as io.github.ribato22/volante
(a validated server.json manifest plus a publish-mcp.yml GitHub Actions workflow
that re-publishes it via OIDC on each release). Directories such as mcp.so,
PulseMCP, and Glama index from it.
As a Claude Code plugin. This repo also doubles as a plugin marketplace — one command wires the
MCP server and a /volante:run slash command into Claude Code:
/plugin marketplace add ribato22/volante
/plugin install volante@volante
Requires uv on your PATH (the plugin launches the server with
uvx). See plugins/volante/.
Other MCP clients (Codex, Cursor, Windsurf, Gemini CLI, Cline). These don't have a plugin marketplace — they consume MCP servers via config. Point them at the same launch command:
OpenAI Codex CLI — codex mcp add volante -- uvx --from "volante[mcp]==0.5.0" volante-mcp
(writes an [mcp_servers.volante] block to ~/.codex/config.toml; add providers with repeated
--env KEY=VALUE).
Gemini CLI — gemini mcp add volante uvx -- --from "volante[mcp]==0.5.0" volante-mcp
(the -- is required because Volante's first arg is --from).
Cursor / Windsurf / Cline / Roo — add the standard mcpServers entry to the client's MCP
config (~/.cursor/mcp.json, ~/.codeium/windsurf/mcp_config.json, or the Cline settings):
{ "mcpServers": { "volante": {
"command": "uvx",
"args": ["--from", "volante[mcp]==0.5.0", "volante-mcp"],
"env": { "CLAUDE_CODE_ENABLED": "1", "ANTHROPIC_API_KEY": "sk-ant-..." }
} } }
Set your providers in each client's env block (CLAUDE_CODE_ENABLED, CODEX_ENABLED,
ANTHROPIC_API_KEY, OPENAI_COMPAT_*).
Claude Desktop & Smithery — MCPB bundle. A one-file MCPB bundle is attached to each
release as volante-<version>.mcpb. Open it in Claude
Desktop for a one-click install (it shows a provider-config UI). Or publish it to
Smithery as a local server — because Volante is stdio (not a hosted HTTPS
endpoint), that's the CLI bundle path, not the "publish a URL" web form:
# download volante-<version>.mcpb from the release, then (needs a Smithery API key):
npx -y @smithery/cli mcp publish ./volante-<version>.mcpb -n <your-namespace>/volante
The bundle wraps uvx --from "volante[mcp]==0.5.0" volante-mcp, so it runs locally and your
subscription CLIs + API keys work as usual (needs uv on PATH).
Set environment variables for any subset. Every configured model becomes part of the user's model
inventory considered by the router; the Anthropic > OpenAI-compat > Kimi > Ollama priority only
chooses the default planner/synthesizer. See .env.example for the full list.
| Provider | Env | Access |
|---|---|---|
| Anthropic (Claude) | ANTHROPIC_API_KEY | Paid API (console.anthropic.com) |
| Generic OpenAI-compatible | OPENAI_COMPAT_BASE_URL, OPENAI_COMPAT_MODEL (+_KEY/_NAME/_CONTEXT/…) | Any OpenAI-compatible endpoint |
| Moonshot / Kimi | MOONSHOT_API_KEY | Paid API |
| Ollama | OLLAMA_BASE_URL | Local & free |
Volante never guesses which remote models an account owns. Provider catalog and entitlement APIs are inconsistent, can require extra permissions, and do not prove that a model is currently usable. Instead, the routing inventory is explicit and auditable:
# Comma-separated lists; the singular forms remain backward compatible.
export ANTHROPIC_MODELS=claude-opus-4-8,claude-sonnet-4-5
export ANTHROPIC_NAMES=anthropic/opus,anthropic/sonnet
export MOONSHOT_MODELS=kimi-k3,kimi-k2.6
export OLLAMA_MODELS=qwen2.5-coder:14b,llama3.2
export CLAUDE_CODE_MODELS=opus,sonnet
export CODEX_MODELS=gpt-5-codex,gpt-5-codex-mini
uv run volante --list-models
uv run volante --list-models --json
*_NAMES is optional and, when supplied, must contain exactly one canonical id per wire model.
All configured models are registered; no default seed is silently added. The inventory command is
offline: “configured/detected” does not claim live entitlement, quota, or service availability.
At execution time, an explicit model/deployment-not-found or model-access denial marks only that
candidate unavailable. A generic authentication or endpoint failure marks the provider unavailable.
Volante records either fallback event and can try the next ranked model; malformed or semantically
invalid requests still fail fast instead of blindly calling every provider.
Without measured evidence, quality is deliberately a prediction based on declared strengths,
tier, context/output headroom, and task difficulty. Volante does not pad that prediction: a scoring
component that says the same thing about every eligible model (task_fit when no model declares a
specialized strength, reliability when no profile is configured) is given zero weight and its
share is redistributed to the components that actually carry information. The route trace names the
components it dropped, so a tier-driven ranking reads as exactly that instead of hiding behind a
45-point “task fit” constant.
Two levers turn those components back on.
1. Declare what a model is actually good at. Beyond the required strength for a task type
(coding for code, reasoning for the rest), these optional tags raise task_fit and let peers
of the same tier be ranked apart:
| Task type | Optional strength tags that improve fit |
|---|---|
code | software_engineering, debugging, instruction_following |
research | research, grounding, long_context |
write | writing, creativity, instruction_following |
analyze | analysis, math, long_context |
Set them per provider slot with the *_STRENGTHS env vars (for example
ANTHROPIC_STRENGTHS=coding,reasoning,software_engineering,debugging) or per model in the
overrides file. These are your declarations, not vendor claims Volante bakes in.
2. Calibrate from measurements you own. Convert scores you actually observed into a strict profiles file — no provider calls, so it costs nothing:
# measurements.json — one entry per observed run; null means "produced nothing usable":
# {"anthropic/opus": {"code": [1.0, 0.8], "write": [0.9]},
# "ollama/qwen": {"code": [0.4, null]}}
uv run volante --calibrate measurements.json --calibrate-out quality-profiles.json
export VOLANTE_QUALITY_PROFILES_FILE=$PWD/quality-profiles.json
--calibrate averages per task type. It emits overall_score only when every task type was
measured, because the router applies that field to every task type — including ones your
measurements never touched — so deriving it from a partial sample would let your coding evidence
stand in for research ability nobody looked at. With partial coverage the field is omitted, the
router falls back to the coarse declared tier, and its trace says so instead of citing a profile
that never looked. When coverage IS complete it macro-averages the per-type means, so an unbalanced
sample (30 code runs, one write run) describes the model rather than your sampling.
confidence follows the weakest-sampled task type and is capped below 1.0 — the router
applies one confidence to every task type, so unrelated runs must not make a single observation
read as certain. reliability_score is emitted only when you recorded null runs: deriving it
from low scores would make the reliability component a copy of the quality component. Model ids must
exist in your configured inventory (the loader is strict), and the output replaces rather than
merges, so calibrate every model you care about in one file.
Where those measurements come from. eval/calibrate_models.py runs the eval suite's baseline
arm — one model, one call, no orchestration — across several models and writes the measurements
file for you. It spends real money (models x goals x k calls):
uv run python -m eval.calibrate_models --models gpt-4.1-nano,gpt-4o-mini,gpt-4.1 --k 3
uv run volante --calibrate measurements.json --calibrate-out quality-profiles.json
measurements.json and quality-profiles.json in this repo are a real run of exactly that
(2026-07-29, 81 calls). Read them as a worked example, not as defaults: they describe three OpenAI
models you probably do not have, and the router only loads a profile you point it at.
What the measurements establish. Three OpenAI models produced a clean monotonic gradient on
code — gpt-4.1-nano 0.951, gpt-4o-mini 0.990, gpt-4.1 1.000 — which agreed with the tier
order the router already used, so calibrating within one family confirmed the heuristic rather than
overturning it. Measuring a second family is what changed the picture:
| model | code | analyze |
|---|---|---|
openai/gpt-4.1 | 1.000 | 0.417 |
openai/gpt-4o-mini | 0.990 | 0.167 |
openai/gpt-4.1-nano | 0.951 | 0.183 |
glm/glm-4.5-flash | 0.704 | 0.389 |
glm-4.5-flash is last at code and second at analysis; gpt-4o-mini is nearly best at code
and worst at analysis — 0.29 the wrong way on one task type, 0.22 the right way on the other,
against a 0.016 within-family wobble that was noise.
How much weight that carries, stated plainly. The code figure is 9 goals x k=3 per model. The
analyze figure is one goal — eval/tasks_text.py currently has a single item — at k=3 to 5,
and every score sits in the lower half of that rubric's range. A swap this size on a single item is
suggestive, not established; treat it as a reason to measure your own inventory, not as a result to
cite. Whether models from different labs have genuinely complementary strengths is the assumption
per-task routing rests on, and this is the first evidence here pointing at it — one goal's worth.
With that evidence loaded and the strongest model excluded, the router picks gpt-4o-mini for
code and glm-4.5-flash for analyze — different models for different work, from measurement
rather than from a tier constant. When the strongest model IS available it still wins both, because
it is genuinely best at both; the swap shows up in everything below it, which is what a cost or
quota objective actually chooses among.
Two limits worth knowing before you trust a profile of your own:
gpt-4.1 scored 27/27 perfect, so nothing above it can be measured and
the gradient is compressed into the top 5%. At k=1 the weakest model scored a perfect 1.000
by luck; only k=3 separated them. Calibrate at k>=3, and expect a task set that everything
passes to tell you nothing.code. That used to leak: overall_score was derived from whatever
was measured and then applied to every task type, so coding evidence silently became a research
claim. It no longer does — a general claim now requires complete coverage — and the effect is
visible in the scores: with a code-only profile loaded, research ranks exactly as it does with
no profile at all, while code moves. eval/tasks_text.py adds goals for the other types.The profile file format is the same one you can write by hand:
{
"anthropic/opus": {
"task_scores": {"code": 0.96, "research": 0.94, "write": 0.91, "analyze": 0.95},
"overall_score": 0.95,
"reliability_score": 0.98,
"confidence": 0.9,
"source": "team-eval-2026-07"
},
"ollama/qwen2.5-coder:14b": {
"task_scores": {"code": 0.78},
"overall_score": 0.70,
"reliability_score": 0.86,
"is_local": true,
"confidence": 0.8,
"source": "team-eval-2026-07"
}
}
Save that strict JSON outside source control and set
VOLANTE_QUALITY_PROFILES_FILE=/absolute/path/to/quality-profiles.json. Scores are normalized
0..1; supported task keys are code, research, write, and analyze. Unknown model ids,
unknown fields, duplicate JSON keys, invalid/non-finite scores, and credential-like extra fields
fail closed. A decision trace records the evidence source and caveat rather than claiming an
empirically universal winner.
The repository includes a 3-arm evaluation harness, but Volante does not yet publish representative cross-provider empirical benchmark results or automatically calibrate quality profiles. Treat the configured scores as user-owned evidence, validate them against your own task distribution, and recalibrate them as models or endpoints change.
Plural family settings share their provider-level defaults. When two models in the same family have different hard capabilities, limits, prices, or tiers, declare those per canonical model id in a second strict JSON file:
{
"anthropic/haiku": {
"strengths": ["reasoning"],
"context_window": 200000,
"max_output_tokens": 64000,
"supports_tools": true,
"cost_per_1k_in": 0.001,
"cost_per_1k_out": 0.005,
"tier": 2
},
"anthropic/opus": {
"strengths": ["coding", "reasoning", "long_context"],
"context_window": 1000000,
"max_output_tokens": 128000,
"supports_tools": true,
"cost_per_1k_in": 0.005,
"cost_per_1k_out": 0.025,
"tier": 4
}
}
Set VOLANTE_MODEL_OVERRIDES_FILE=/absolute/path/to/model-overrides.json. Overrides may contain only
the seven fields shown above; model ids must already exist in the configured inventory. Quality
profiles carry soft, evaluation-derived evidence, while model overrides carry hard/economic
metadata used for eligibility, projection, routing, and accounting. Neither file accepts secrets.
Agentic tool availability is also explicit. run_python is offered only when an isolating sandbox
is available (see Security); set
VOLANTE_FETCH_ALLOWLIST=example.com,docs.python.org to enable fetch_url, and set
VOLANTE_READ_ROOT=/absolute/path/to/trusted/files to enable read_file. The supervisor declares
required_tools per agentic task, and Volante rejects a plan or execution when those tools are not
enabled or were never actually invoked. The hostname allowlist is not a complete SSRF defense
against DNS rebinding/private resolution, and VOLANTE_READ_ROOT should point to a trusted tree
without adversarial concurrent symlink changes.
Subscription CLI agents are opt-in and consume your interactive quota. Scraping claude.ai / ChatGPT is not built (ToS, fragile, ban risk). Instead Volante can drive the official headless CLIs you're already logged into — Claude Code (
claude -p) and Codex (codex exec) — with no API key. This is off by default (CLAUDE_CODE_ENABLED=1/CODEX_ENABLED=1plus the CLI installed) and is never used by the eval. Honest caveat:claude -pandcodex exectoday draw from the same interactive subscription pool as the chat apps (not a separate/metered bucket), so a full orchestration run — and especially the 3-arm eval — can burn your Claude Code / Codex allowance and trip a mid-run hard-pause. Volante reportscredit_usd(subscription value consumed) separately frombilled_usd(cash), routes only hard/high-tier tasks to subscription (bulk work goes to local/free-tier), and caps physical subscription calls per run (VOLANTE_MAX_SUBSCRIPTION_CALLS, default 16, including the subscription-planner compatibility preflight, planning, retries, worker/agent turns, and synthesis). Under the defaultqualityobjective, any eligible subscription model may be selected; the hard-task reservation behavior applies only to--prefer cash_protect_quota.Billing surface moves — re-verify before trusting it. Whether
claude -pbills against the subscription pool vs. a metered API-rate credit bucket has flipped several times in months (announced 2026-06-15, then paused; still paused as of 2026-07-22). When Anthropic next announces a billing change, repeat the live gate indocs/claude-code-live-gate.mdand re-check the Help Center banner, then update the "verified" date recorded there.
Free, high-intelligence option — Google AI Studio (Gemini Flash), via the generic slot:
export OPENAI_COMPAT_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
export OPENAI_COMPAT_KEY=<ai-studio-key> # aistudio.google.com/apikey
export OPENAI_COMPAT_MODEL=gemini-flash-latest # pick a current model from the endpoint's /models
export OPENAI_COMPAT_NAME=google/gemini-flash
uv run python demo.py orchestrate
The generic slot defaults to context 128k, output 8k, tool support off, and cost 0. Set
OPENAI_COMPAT_TOOLS=true only after confirming function-calling support, and set the provider's
actual costs/tier/strengths; Volante registers that ModelInfo so routing and accounting use the
declared metadata rather than a hidden seed.
Several models/providers at once — add OPENAI_COMPAT_2_*, OPENAI_COMPAT_3_*, … (each with
its own model_id / pricing / context), or use the plural model lists above for Anthropic,
Moonshot, Ollama, Claude Code, and Codex. Slots may point to different providers or to several
models from the same compatible endpoint. For example, configure Gemini plus Groq so the supervisor
plans on Gemini while a Groq model runs parallel workers. See .env.example.
demo.py eval runs a 3-arm comparison over 5 composite coding goals: baseline (one strong
model, one shot), orchestration (the full engine), and agentic-single (one model + a
run_python loop, no decomposition). Each goal is scored by a hidden reference test.
The scorer runs the model's generated solution.py in a subprocess under process + filesystem
separation: a trusted runner drives the untrusted solution in a separate process that never sees
the expected outputs (nonce-authenticated RPC), so a solution must actually compute correct answers —
it cannot fake a passing score.
Read the verdict together with the warnings the harness emits:
WARNING: some costs are estimated … — a provider returned no usage; cost comparison is soft.WARNING: agentic arm failed N run(s) … — a 0.0 may be infra/provider failure, not capability.WARNING: goal(s) […] produced NO trusted result … — the reference runner itself is broken; those
scores are harness artifacts, not real zeros.Volante is alpha software; its isolation guarantees are deliberately scoped and documented.
VOLANTE_SANDBOX set, Volante
probes for a running Docker daemon: if one answers, run_python executes in a container
(--network none, read-only root, dropped capabilities, cgroup limits). If none answers,
run_python is not offered at all — the planner is told it cannot execute code — rather than
quietly running model-written code with your files and network. The subprocess Sandbox protects
against accidents, not adversaries (host network and disk stay reachable), so it is now an
explicit opt-in: VOLANTE_SANDBOX=subprocess, which prints a warning on every start. An unknown
VOLANTE_SANDBOX value is rejected instead of silently downgrading.fetch_url (hostname allowlist, no redirects, bounded body)
and read_file (resolved-path root check, bounded read) run in the trusted orchestrator so
sandboxed code stays network-isolated. The allowlist does not defeat DNS rebinding/private
address resolution, and the root check is not race-proof against a hostile symlink swap; use
trusted domains and trusted local trees. Prompt-injection containment holds only under Docker.setsid() can still escape the killpg group
(the wall-clock timeout still bounds the run). It is process isolation, not a security sandbox for
arbitrary hostile code.src/volante/ # engine (importable package: `volante`)
providers/ # Anthropic + OpenAI-compatible adapters, FakeProvider
tools/ # Sandbox, DockerSandbox, run_python, fetch_url, read_file
eval/ # goals, 3-arm harness, forgery-resistant scorer, runner
examples/ # small runnable library-API scripts (incl. a no-key FakeProvider demo)
webui/ # optional FastAPI + SSE web UI (uv run python -m webui)
tests/ # 800+ tests (unit + opt-in integration)
docs/ # internal design/build records — see docs/README.md; not user docs
demo.py # end-to-end demo (orchestrate | agentic | eval)
uv run pytest uses FakeProvider and local
subprocesses; integration tests that touch the network/Docker are marked integration and skipped
by default (uv run pytest -m integration to opt in).uv run ruff check . (line length 100; E,F,I,UP,B).Volante is an alpha, transparent, user-owned model router and orchestration control plane. It is usable today through its CLI, library, Web UI, and MCP server for inventories and providers that the user explicitly configures. It is not a managed model gateway, an automatic entitlement-discovery service, or a guarantee that its predicted fit is empirically optimal. Interfaces may still evolve.
For production use, validate Volante's routing against a representative workload, supply and maintain your own quality evidence, choose the documented isolation mode for the threat model, and retain an application-level recovery path. The framework-free implementation is intentional: supervisor, router, projector, and blackboard behavior stays inspectable instead of being hidden behind LangChain, LiteLLM, or CrewAI abstractions.
MIT © 2026 ribato.
FAQs
Volante — a transparent, user-owned model router and orchestration control plane.
We found that volante demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
A misconfiguration gave three Anthropic models internet access, and one, believing it was in a simulation, shipped a credential-stealing package to PyPI.
Security News
/Company News
Socket has joined the new Composer and Packagist sponsorship program as a launch sponsor, supporting the team that keeps PHP's package ecosystem secure.
Research
/Security News
Benign-looking npm packages split malicious functionality across a dependency chain that deploys a cross-platform RAT targeting Alibaba developers.