@phuetz/code-buddy

Code Buddy

Your AI-Powered Development Tool & Personal Assistant

The open-source AI coding agent that runs free, on your own machine.

Code, commands, web, voice, and vision — from your terminal, a desktop app, your phone, or a 24/7 autonomous service. Use local Ollama models at $0, or bring any of 15 providers. No lock-in.

_{A local model reasons, then uses a tool to create a real file — ~$0.0001, no cloud. More demos →}

• 🆓 Free & local-first — runs entirely on local Ollama ($0), or any of 15 providers (Claude, GPT, Grok, Gemini, …) with auto-failover. Or log in with ChatGPT Plus/Pro for a flat-fee brain — no API metering.
• 🧠 Reasoning you can watch — local models think step-by-step on screen, then act — see the live captures.
• 🖥️ Runs everywhere — terminal TUI, the Cowork desktop app, an HTTP/WebSocket server, your phone, or a 24/7 background service — all on the same core engine.
• 🤝 Multi-AI Fleet — multiple peers observe each other live and call each other's models & read-only tools (peer.chat / peer.tool.invoke) across your network.
• 🤖 Autonomous — buddy autonomy install runs a self-driving service that claims & executes tasks free-first on local models, 24/7.
• 🛠️ ~110 tools, skills & MCP — edit, shell, web search, browser, PDFs/Office, a skills marketplace, and MCP connectors to extend it.
• 👁️ Personal companion — bidirectional voice, opt-in camera/presence (MediaPipe), persistent memory, and 20+ messaging channels.

Quick Start | Cowork + Companion | Features | FAQ | Documentation | Contributing

What is Code Buddy?

Code Buddy is an open-source multi-provider AI coding agent with a terminal UI, HTTP/WebSocket server, and Cowork desktop app. It supports 15 LLM providers with automatic failover and per-provider circuit breakers. It works as both a development tool (reads files, writes code, runs commands, creates PRs, plans complex tasks) and a personal companion (bidirectional voice conversation, durable memory, opt-in camera perception, screen/presence context, push notifications via 20+ messaging channels, and 24/7 background operation). With buddy login, a ChatGPT Plus / Pro subscription can become the flat-fee brain of the system without API-key metering.

In action

Free local AI, with the reasoning on screen. A local Ollama model (qwen3.6:35b-a3b) thinks through a task, then uses tools to do it — no cloud, ~$0.0001. Unedited captures from the Cowork desktop app:

Reasoning chat — thinks step-by-step, then answers · local · ~$0.0001

Real task — reasons, uses the file tool, confirms the artifact · local · ~$0.0001

More desktop demos (Fleet, Autonomy, Companion, …): cowork/readme.md.

ChatGPT Pro / Plus subscription login — buddy login, sign in once with your ChatGPT account, then chat with gpt-5.5 directly from the terminal. No API key, cost reported as $0.0000 (flat-fee plan).

Interactive TUI + tool calling. The agent reads project context, calls tools in parallel (web_search ×2 for the screenshot below), and streams the synthesised answer.

Self-audit. Asked to find a bug in its own integration code, gpt-5.5 reads src/codebuddy/providers/provider-chatgpt-responses.ts, identifies a stale-variable issue (mutated body.model not propagated), and proposes the exact fix:

More captures + walk-through: docs/screenshots/.

Quick Start

# Install — from source (recommended during the 1.0 release-candidate phase: gets the latest)
git clone https://github.com/phuetz/code-buddy.git
cd code-buddy && npm install && npm run build && npm link   # exposes `buddy` globally

# Or install the published release from npm
# ⚠️ during rc, the npm release can lag the source — prefer from-source for the newest features
npm install -g @phuetz/code-buddy

# Option A — bring your own API key
export GROK_API_KEY=your_api_key   # or GEMINI_API_KEY / OPENAI_API_KEY / ANTHROPIC_API_KEY
buddy

# Option B — log in with your ChatGPT Plus / Pro subscription (no API key needed)
buddy login                         # opens browser for OAuth → tokens persisted
buddy whoami                        # ✅ connected · [e-mail] · Plan: pro
buddy                               # auto-routes to gpt-5.5 via the Codex backend, cost $0.0000

# Or with a specific task
buddy --prompt "analyze the codebase structure"

# Full autonomy
buddy --yolo

See Getting Started for installation options, headless mode, session management, and typical workflows. The ChatGPT Codex OAuth flow is documented with screenshots in docs/screenshots/. The desktop cockpit has a public overview in Cowork Desktop.

Cowork Desktop + Buddy Companion

Cowork is the desktop cockpit for Code Buddy: chat, tools, traces, workflows, settings, permissions, models, MCP connectors, skills, artifacts, and companion controls all run against the same core agent as the CLI. The Code Buddy settings panel can probe the local backend, start it when needed, discover models, and route Cowork turns through the embedded engine or a configured server endpoint.

# First-time identity and flat-fee brain route
buddy login
buddy companion setup

# Local backend for Cowork, Fleet, and OpenAI-compatible clients
buddy server --port 3000

# Launch the desktop app from an installed build
buddy gui
# or
buddy desktop

# Source checkout dev loop
npm install
npm run build
npm run dev:gui

Buddy companion commands are available in both the CLI and Cowork panel:

buddy companion status
buddy companion self
buddy companion evaluate
buddy companion radar
buddy companion impulses
buddy companion missions sync
buddy companion missions run-next
buddy companion safety recent
buddy companion camera status
buddy companion camera snapshot
buddy companion percepts recent

The camera bridge is explicit and local: snapshots are opt-in, percepts are append-only under .codebuddy/companion/, and Cowork uses MediaPipe Tasks Vision for face, hand, finger-tip, and pose signals. Face enrollment/presence recognition lives in Cowork's presence bridge and uses the local MediaPipe/Buffalo_S pipeline.

For low-latency voice experiments, Cowork can route STT/TTS through a local Kyutai DSM / moshi-server endpoint while keeping faster-whisper and Piper as fallbacks:

$env:COWORK_VOICE_PROVIDER='kyutai'
$env:COWORK_KYUTAI_URL='ws://127.0.0.1:8080'
npm run dev:gui

Use Buddy companion's Inspect voice action to probe the active route, Kyutai STT/TTS websocket reachability, ffmpeg availability, and the Piper/faster-whisper fallback state from the Electron app.

From source, Cowork requires Node.js >=22 in cowork/; the root CLI still supports Node.js >=18.

Features

Category	Highlights	Docs
AI Providers	15 providers (Grok, Claude, GPT, Gemini, Ollama, LM Studio, AWS Bedrock, Azure, Groq, Together, Fireworks, OpenRouter, vLLM, Copilot, Mistral), circuit breaker, model pairs	providers.md
Tools	~110 tools with RAG selection, multi-strategy edit matching, Codex-style apply_patch, streaming, BM25 tool search, code exec sandbox	tools-reference.md
Commands	190+ slash commands, CLI subcommands (/dev, /pr, /lint, /switch, /think, /batch, /watch, /conflicts, /vulns, /replace)	commands.md
Cowork Desktop	Electron cockpit, embedded Code Buddy engine, backend health/start controls, model settings, permission rules, visual workflows, traces, artifacts, MCP/skills/plugin management	Cowork Desktop, cowork/readme.md, cowork/ARCHITECTURE.md
Buddy Companion	ChatGPT-backed identity, voice/TTS, proactive check-ins, self-evaluation, competitive radar, mission board, learned routines, safety ledger, local percept journal	commands.md
Vision & Presence	Opt-in webcam snapshots, MediaPipe face/hand/pose/finger-tip analysis, local face enrollment, presence state for the agent	cowork/ARCHITECTURE.md
Agents	Multi-agent orchestration (5-tool API), 8 specialized agents, SWE agent, planning flow, A2A protocol, batch decomposition, agent teams	agents.md
Reasoning	Tree-of-Thought + MCTS (4 depth levels), extended thinking, auto-escalation, /think command	reasoning.md
Security	Guardian Agent (AI risk scoring), OS/Docker/OpenShell sandbox, SSRF guard, secrets vault, write policy, exec policy, loop detection, omission detection, output sanitizer	security.md
Channels	20+ messaging channels (Telegram, Discord, Slack, WhatsApp, Signal, Teams, Matrix, IRC, and more), DM pairing, send policy	channels.md
Context Engine	Smart compression, tool output masking, image pruning, transcript repair, pre-compaction flush, restorable compression, JIT context, importance-weighted window	context-engine.md
Fleet & Autonomy	Peer-to-peer hub (peer.chat / peer.tool.invoke / peer_delegate), A2A + ACP + MCP interop, 24/7 autonomous service (buddy autonomy install), event-driven daemon, claim TTL/lease, DAG task dependencies, workers→verifier→synthesizer swarm, free-first local→Tailscale→paid model tier	fleet-guide.md
Infrastructure	HTTP server (OpenAI-compatible), WebSocket gateway, daemon mode, cron, device nodes, canvas/A2UI, 6 cloud deploy configs, MCP, plugins	infrastructure.md
Configuration	Env vars, TOML config with profiles, model-aware limits, per-agent params, i18n (6 locales), personas	configuration.md
Development	TypeScript strict, Vitest (27,334 tests), ESM, middleware pipeline, facade architecture	development.md

Additional Capabilities

• Voice: 7 TTS providers, wake word detection, voice-to-code pipeline, hands-free companion conversation
• Companion Loop: readiness checks, self-state snapshots, self-evaluation, competitive radar, impulses, missions, routines, and safety review
• Vision: local camera snapshots, MediaPipe face/hand/pose/finger-tip percepts, face enrollment, and presence state for context-aware collaboration
• Memory: Persistent + semantic + prospective + decision + coding style memory, ICM cross-session memory
• Knowledge: Knowledge base injection, 40 bundled skills, self-authoring skills at runtime
• Git Workflow: Auto-commit (Aider-style), /pr creation, merge conflict resolver, ghost snapshots
• Code Intelligence: LSP rename/refactor, auto-import, bug finder (25+ patterns, 6 langs), OpenAPI generator, log analyzer
• IDE Integration: VS Code extension (diff view, inline edit, model switch), JetBrains plugin, LSP server
• Inline Context: @web, @git, @terminal mentions for contextual references
• Workflows: Lobster typed DAG engine with approval gates, pause/resume tokens

Documentation

Document	Description
Getting Started	Prerequisites, install, first run, headless mode, session management
Application Validation Guide	Full-app validation matrix with safe commands, evidence, and screenshot rules
Providers	All 15 providers, connection profiles, model pairs, circuit breaker
Tools Reference	Tool categories, RAG selection, edit matching, apply_patch, streaming
Commands	All slash commands, CLI subcommands, companion commands, global flags
Cowork Desktop	GitHub-visible overview, ChatGPT gpt-5.5 route, real validation commands, screenshot privacy policy
Autonomous Coding + Cowork Progress	End-to-end autonomous task execution, RunStore artifacts, and Cowork progress tracking
Cowork README	Desktop installation, features, source build, sandbox modes
Cowork Architecture	Electron contexts, bridges, embedded engine, persistence, runner model
Agents	Multi-agent orchestration, roles, SWE agent, planning flow, A2A
Reasoning	Extended thinking, Tree-of-Thought, MCTS, /think command
Security	Permission modes, Guardian Agent, sandboxing, SSRF, secrets vault
Channels	20+ messaging channels, DM pairing, send policy
Context Engine	Compression, tool output masking, JIT context, pre-compaction flush
Infrastructure	HTTP server, WebSocket gateway, daemon, cron, deploy, plugins
PdfCommander MCP Integration	Register the PdfCommander headless MCP server in mcp.json, verify with buddy mcp test, drive PDF ops from the agent
Fleet Guide	Multi-AI hub — /fleet listen + /fleet send peer.chat, env-driven multi-provider auto-detect, Tailscale lab examples, autonomous fleet protocol
Hermes / OpenClaw Parity	Where Code Buddy stands vs Hermes Agent & OpenClaw — shipped capabilities, externally-gated features, and the one open code gap
Configuration	Environment variables, TOML config, project settings, model limits
Development	Build, test, architecture, coding conventions, adding tools

Validation Snapshot

Latest local verification for the Cowork + Buddy companion loop (2026-05-24):

npm run typecheck
cd cowork && npm run typecheck
cd cowork && npm run build:e2e
cd cowork && npx vitest run tests/kyutai-bridge.test.ts tests/voice-bridge.test.ts tests/tts-bridge.test.ts
cd cowork && npx playwright test e2e/cowork-smoke.spec.ts --reporter=line --workers=1
cd cowork && npx playwright test e2e/companion-panel.spec.ts --reporter=line --workers=1
cd cowork && npx playwright test e2e/recent-features-smoke.spec.ts e2e/companion-panel.spec.ts --workers=1
cd cowork && npx playwright test e2e/codebuddy-settings.spec.ts e2e/recent-features-smoke.spec.ts e2e/companion-panel.spec.ts e2e/companion-live.spec.ts --reporter=line --workers=1
cd cowork && npx vitest run

Result: root and Cowork typechecks passed, Vite E2E build passed with existing chunk/dynamic-import warnings, Kyutai/voice bridge unit tests passed (31 tests), focused companion panel Playwright passed (1 test), recent-features IPC smoke passed (9 tests), Cowork smoke passed (29 tests), focused companion/settings/recent-feature Playwright suite passed (11 passed, 1 live hardware test skipped unless COWORK_LIVE_COMPANION=1), and Cowork Vitest passed (205 files, 1318 tests).

Contributing

git clone https://github.com/phuetz/code-buddy.git
cd code-buddy
npm install
npm run dev          # Development mode
npm test             # Run all tests
npm run validate     # Lint + typecheck + test (run before committing)

See Development for architecture details and coding conventions.

License

MIT License - see LICENSE for details.

Report Bug | Request Feature | Star on GitHub

_{Multi-AI: Grok | Claude | ChatGPT | Gemini | LM Studio | Ollama | AWS Bedrock | Azure | Groq | Together | Fireworks | OpenRouter | vLLM | Copilot | Mistral}