backbeat

Backbeat: AI Coding Agent Orchestration at Scale

Why Backbeat

Every AI coding agent is a single-threaded workhorse. One task, one agent, one repo at a time. That is a massive waste of computing power.

Backbeat turns your machine into an AI development powerhouse. Run dozens of agents in parallel. Chain them into pipelines. Score their output and iterate until optimal. Three models, unlimited parallelism, one orchestrator.

Karpathy Optimization Loops. The only production-grade framework that implements optimization loops with eval scoring for AI coding agents. Run a task, score the output with an eval script, iterate until optimal. Retry strategy (run until exit condition passes) or optimize strategy (minimize/maximize any metric). This is the Karpathy loop for coding agents, and nothing else out there does it.

Multi-Agent Pipelines. Chain sequential steps with dependencies. Schedule them on cron. Each step can use a different agent: Claude for architecture, Codex for implementation, Gemini for review.

DAG Task Dependencies. Full directed acyclic graph with cycle detection, failure cascading, and session continuation. Downstream tasks receive checkpoint context from completed dependencies.

Autoscaling Workers. Dynamic worker pool monitors CPU and memory in real-time. Spawns agents when resources are available. No artificial limits.

Crash-Proof Persistence. SQLite with WAL mode. Automatic state recovery. Kill the process, reboot your machine, come back tomorrow. Everything is exactly where you left it.

Three AI models. Unlimited parallelism. One orchestrator.

Features

• Karpathy Optimization Loops that run, score with eval scripts, and iterate until optimal. Retry strategy (exit code = done) or optimize strategy (minimize/maximize scoring). Pipeline loops repeat multi-step workflows.
• Multi-Agent Orchestration to delegate to Claude, Codex, or Gemini with per-task agent selection
• DAG Task Dependencies with cycle detection, failure cascading, and session continuation via checkpoint context injection
• Scheduled Pipelines that chain sequential steps on cron or one-time schedules with per-step priority, working directory, and agent overrides
• Autoscaling Workers with real-time CPU/memory monitoring and dynamic worker pool
• Crash Recovery with SQLite WAL mode, automatic state recovery, and checkpoint-based task resumption
• Event-Driven Architecture with specialized handlers that eliminate race conditions

See FEATURES.md for the complete list.

Quick Start

Prerequisites

• Node.js 20.0.0+
• npm 10.0.0+
• An AI coding agent CLI installed (e.g., claude, codex, gemini-cli)

System Requirements

Minimum (for development/testing):

• 8+ CPU cores
• 16GB RAM
• 100GB SSD

Recommended (for production):

• 32+ CPU cores
• 64GB+ RAM
• 500GB+ NVMe SSD
• Dedicated Linux server (Ubuntu 22.04+)

Installation

Add to your project's .mcp.json:

{
  "mcpServers": {
    "backbeat": {
      "command": "npx",
      "args": ["-y", "backbeat", "mcp", "start"]
    }
  }
}

Restart your MCP client to connect to Backbeat.

Usage

MCP Tools

Once configured, use these MCP tools:

Tool	Description	Usage
DelegateTask	Submit tasks to background instances	DelegateTask({ prompt: "...", priority: "P1" })
TaskStatus	Get real-time task status	TaskStatus({ taskId })
TaskLogs	Stream or retrieve execution logs	TaskLogs({ taskId })
CancelTask	Cancel tasks with resource cleanup	CancelTask({ taskId, reason })
RetryTask	Retry a failed or completed task	RetryTask({ taskId })
ScheduleTask	Schedule recurring or one-time tasks	ScheduleTask({ prompt: "...", scheduleType: "cron", cronExpression: "0 2 * * *" })
ListSchedules	List schedules with optional status filter	ListSchedules({ status: "active" })
ScheduleStatus	Get schedule details and execution history	ScheduleStatus({ scheduleId })
CancelSchedule	Cancel an active schedule (optionally cancel in-flight tasks)	CancelSchedule({ scheduleId, reason, cancelTasks? })
PauseSchedule	Pause a schedule (resumable)	PauseSchedule({ scheduleId })
ResumeSchedule	Resume a paused schedule	ResumeSchedule({ scheduleId })
ResumeTask	Resume a failed/completed task with checkpoint context	ResumeTask({ taskId, additionalContext? })
CreatePipeline	Create sequential task pipelines	CreatePipeline({ steps: [...] })
SchedulePipeline	Create recurring/one-time scheduled pipelines	SchedulePipeline({ steps: [...], cronExpression: "0 9 * * *" })
CreateLoop	Create iterative loops (retry or optimize strategy)	CreateLoop({ prompt: "...", strategy: "retry", exitCondition: "npm test" })
LoopStatus	Get loop details and iteration history	LoopStatus({ loopId })
ListLoops	List loops with optional status filter	ListLoops({ status: "running" })
CancelLoop	Cancel an active loop (optionally cancel in-flight tasks)	CancelLoop({ loopId, cancelTasks: true })
PauseLoop	Pause an active loop (graceful or force)	PauseLoop({ loopId, force?: true })
ResumeLoop	Resume a paused loop	ResumeLoop({ loopId })
ScheduleLoop	Schedule a recurring or one-time loop	ScheduleLoop({ prompt: "...", strategy: "retry", exitCondition: "...", cronExpression: "0 * * * *" })

CLI Commands

Command	Description
beat mcp start	Start the MCP server
beat run <task>	Submit new task (fire-and-forget by default)
beat run <task> -f	Submit task and stream output (foreground mode)
beat list / beat ls	List all tasks
beat status [task-id]	Check task status (all tasks if no ID)
beat logs <task-id>	View task output
beat cancel <task-id>	Cancel running task
beat retry <task-id>	Retry a failed or completed task
beat resume <task-id>	Resume a task from its checkpoint
beat schedule create <prompt>	Create a cron or one-time schedule
beat schedule list	List schedules with optional status filter
beat schedule status <id>	Get schedule details and execution history
beat schedule pause <id>	Pause an active schedule
beat schedule resume <id>	Resume a paused schedule
beat schedule cancel <id>	Cancel a schedule
beat pipeline <prompt> ...	Create chained one-time schedules
beat loop <prompt> --until <cmd>	Create a retry loop (run until condition passes)
beat loop <prompt> --eval <cmd>	Create an optimize loop (score-based)
beat loop list	List loops with optional status filter
beat loop status <loop-id>	Get loop details and iteration history
beat loop cancel <loop-id>	Cancel a loop
beat config show|set|reset|path	Manage configuration
beat help	Show help

Task Dependencies

Create workflows where tasks wait for dependencies to complete:

# Step 1: Create build task (fire-and-forget by default)
beat run "npm run build" --priority P1
# → task-abc123

# Step 2: Create test task that waits for build
beat run "npm test" --deps task-abc123
# Task waits for build to complete before running

# Step 3: Create deploy task that waits for tests
beat run "npm run deploy" --deps task-def456
# Execution order: build → test → deploy

Multiple dependencies (parallel execution):

// lint and format run in parallel
const lint = await DelegateTask({ prompt: "npm run lint" });
const format = await DelegateTask({ prompt: "npm run format" });

// commit waits for both to complete
const commit = await DelegateTask({
  prompt: "git commit -m 'Formatted and linted'",
  dependsOn: [lint.taskId, format.taskId]
});

Session continuation (pass output context through dependency chains):

// Build task runs first
const build = await DelegateTask({ prompt: "npm run build" });

// Test task receives build's output/git state in its prompt
const test = await DelegateTask({
  prompt: "npm test",
  dependsOn: [build.taskId],
  continueFrom: build.taskId
});

When continueFrom is set, the dependent task's prompt is automatically enriched with the dependency's checkpoint context (output summary, git state, errors) before execution.

See Task Dependencies Documentation for advanced patterns (diamond dependencies, error handling, failure propagation).

Task Scheduling

Schedule tasks for future or recurring execution:

// Recurring: daily backup at 2am EST
await ScheduleTask({
  prompt: "Backup database to S3",
  scheduleType: "cron",
  cronExpression: "0 2 * * *",
  timezone: "America/New_York",
  missedRunPolicy: "catchup"
});

// One-time: deploy tomorrow at 8am UTC
await ScheduleTask({
  prompt: "Deploy to production",
  scheduleType: "one_time",
  scheduledAt: "2026-02-19T08:00:00Z"
});

Schedule types: cron (5-field expressions) and one_time (ISO 8601 datetime). Missed run policies: skip, catchup, fail. Supports IANA timezones and concurrent execution prevention.

Task Resumption

Resume failed or completed tasks with enriched context from automatic checkpoints:

# Resume a failed task
beat resume task-abc123

# Resume with additional instructions
beat resume task-abc123 --context "Try a different approach this time"

// Via MCP
await ResumeTask({
  taskId: "task-abc123",
  additionalContext: "Focus on the database migration step"
});

Checkpoints are captured automatically on task completion/failure, preserving the last 50 lines of output. Resumed tasks receive the full checkpoint context in their prompt and track lineage via parentTaskId and retryOf fields.

Part of the AI Development Stack

Tool	Role	What It Does
Skim	Context Optimization	Compresses code, test output, build output, and git output for optimal LLM reasoning
DevFlow	Quality Orchestration	18 parallel reviewers, working memory, self-learning, production-grade lifecycle workflows
Backbeat	Agent Orchestration	Orchestration at scale. Karpathy optimization loops, multi-agent pipelines, DAG dependencies, autoscaling

Backbeat scales execution. DevFlow enforces quality. Skim optimizes context. No other stack covers all three.

Architecture

Event-driven system with autoscaling workers and SQLite persistence. Components communicate through a central EventBus, eliminating race conditions and direct state management.

Task Lifecycle: Queued → Running → Completed / Failed / Cancelled

See Architecture Documentation for implementation details.

Configuration

Configuration priority: environment variables > config file > defaults.

Config File

Persistent configuration stored at ~/.backbeat/config.json:

beat config show             # Show resolved configuration
beat config set timeout 300000  # Set a value
beat config reset timeout    # Revert to default
beat config path             # Print config file location

Environment Variables

Variable	Default	Range	Description
TASK_TIMEOUT	1800000 (30min)	1000-3600000	Task timeout in milliseconds
MAX_OUTPUT_BUFFER	10485760 (10MB)	1024-1073741824	Output buffer size in bytes
CPU_CORES_RESERVED	2	1-32	CPU cores reserved for system
MEMORY_RESERVE	2684354560 (2.5GB)	0-68719476736	Memory reserve in bytes
LOG_LEVEL	info	debug/info/warn/error	Logging verbosity

Per-Task Configuration

Override limits for individual tasks:

// Long-running task with larger buffer
await DelegateTask({
  prompt: "analyze large dataset",
  timeout: 7200000,           // 2 hours
  maxOutputBuffer: 104857600  // 100MB
});

// Quick task with minimal resources
await DelegateTask({
  prompt: "run eslint",
  timeout: 30000,             // 30 seconds
  maxOutputBuffer: 1048576    // 1MB
});

Development

Available Scripts

npm run dev        # Development mode with auto-reload
npm run build      # Build TypeScript
npm start          # Run built server
npm run typecheck  # Type checking
npm run clean      # Clean build artifacts

Testing

Tests are grouped to prevent memory exhaustion. npm test is blocked as a safety measure.

# Grouped tests (fast, safe to run individually)
npm run test:core           # Core domain logic (~3s)
npm run test:handlers       # Service handlers (~3s)
npm run test:repositories   # Data layer (~2s)
npm run test:adapters       # MCP adapter (~2s)
npm run test:implementations # Other implementations (~2s)
npm run test:cli            # CLI tests (~2s)
npm run test:integration    # Integration tests

# Full suite (local terminal / CI only)
npm run test:all            # All tests
npm run test:coverage       # With coverage

Project Structure

backbeat/
├── src/
│   ├── core/                # Core interfaces and types
│   ├── implementations/     # Service implementations
│   ├── services/            # Business logic & event handlers
│   ├── adapters/            # MCP adapter
│   ├── bootstrap.ts         # Dependency injection
│   ├── cli.ts               # CLI interface
│   └── index.ts             # Entry point
├── dist/                    # Compiled JavaScript
├── tests/
│   ├── unit/                # Unit tests
│   └── integration/         # Integration tests
└── docs/                    # Documentation

Roadmap

• v0.2.0 - Autoscaling and persistence
• v0.2.1 - Event-driven architecture and CLI
• v0.2.3 - Stability improvements
• v0.3.0 - Task dependency resolution
• v0.3.2 - Settling workers and spawn burst protection
• v0.3.3 - Test infrastructure and memory management
• v0.4.0 - Task scheduling and task resumption
• v0.5.0 - Multi-agent support (Claude, Codex, Gemini)
• v0.6.0 - Architectural simplification + scheduled pipelines
• v0.7.0 - Task/pipeline loops

See ROADMAP.md for detailed plans and timelines.

Troubleshooting

Agent CLI not found

Ensure your agent CLI is in your PATH:

which claude    # or: which codex, which gemini

Server won't start

Check logs in stderr and verify Node.js version:

node --version  # Should be v20.0.0+

Tasks fail immediately

Run in development mode to see detailed logs:

npm run dev

Contributing

Contributions are welcome! Please read our contributing guidelines before submitting PRs.

License

MIT License - see LICENSE file for details

Support

• Report issues: GitHub Issues

Acknowledgments

Built with the Model Context Protocol SDK