testpilot-mcp

testpilot-mcp

Vibe test your app. Just describe what to test.

testpilot-mcp is an MCP server that works alongside @playwright/mcp. You describe the feature to test in plain English — Claude handles the rest: scans your codebase, generates a test plan using your real routes and field names, drives the browser, takes screenshots, records pass/fail results, and writes an HTML report.

You: "test the login flow"
  ↓
testpilot   → scans code, finds /login route + LoginForm fields
            → generates a 12-step test plan with real field names
            → instructs Claude to drive the browser
playwright  → navigates, fills forms, clicks, takes screenshots
testpilot   → records every assertion + screenshot
            → writes HTML report with visual proof

Quick start

1. Add both MCP servers to your AI tool:

Claude Code — ~/.claude/settings.json

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest", "--output-dir", "/tmp/testpilot-screenshots"]
    },
    "testpilot": {
      "command": "npx",
      "args": ["testpilot-mcp"]
    }
  }
}

Cursor — .cursor/mcp.json (or Settings → MCP)

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest", "--output-dir", "/tmp/testpilot-screenshots"]
    },
    "testpilot": {
      "command": "npx",
      "args": ["testpilot-mcp"]
    }
  }
}

Claude Desktop — ~/.claude/claude_desktop_config.json

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest", "--output-dir", "/tmp/testpilot-screenshots"]
    },
    "testpilot": {
      "command": "npx",
      "args": ["testpilot-mcp"]
    }
  }
}

Why --output-dir? testpilot reads screenshots from /tmp/testpilot-screenshots/ to embed them in HTML reports. The --output-dir flag tells Playwright MCP where to save them.

2. Start your dev server (npm run dev, pnpm dev, etc.)

3. Open your AI tool and say:

"Scan this project with testpilot, then test the login flow"

That's it. Claude orchestrates everything.

What Claude does (automatically)

tp_scan_project   → framework: react-router, 7 routes, 4 forms detected
tp_start          → session "Login flow" started
tp_get_context    → reads Login.tsx, useAuth.ts (your actual source)
tp_plan           → generates 12 steps with real field names: email, password

── Claude executes using Playwright MCP ──

browser_navigate        → http://localhost:8080/login
browser_take_screenshot → screenshot-pageload.png  📸
tp_assert               → "Login page loaded" ✅ PASS

browser_fill_form       → { email: "[e-mail]", password: "SecureTest123!" }
browser_click           → submit button
browser_take_screenshot → screenshot-aftersubmit.png  📸
tp_assert               → "Form submitted" ✅ PASS

browser_network_requests → POST /api/auth → 200 OK
tp_assert               → "Auth API returned 200" ✅ PASS

browser_snapshot        → confirms redirect to /clients
tp_assert               → "Redirected to dashboard" ✅ PASS

── Edge cases ──

browser_navigate        → /login (fresh)
browser_click           → submit (empty form)
browser_take_screenshot → screenshot-empty-form.png  📸
tp_assert               → "Empty form shows validation errors" ✅ PASS

tp_end  → PASSED · testpilot-reports/tp-1234567890.html

The HTML report

tp_end writes a self-contained HTML file to ./testpilot-reports/:

• PASSED / FAILED badge at the top
• Project summary: framework, routes scanned, forms, API endpoints
• Assertion table with actual vs. expected values
• Screenshots embedded under each assertion as visual proof
• Full step log with timing

Tools reference

testpilot exposes 6 tools. Playwright MCP handles all browser interactions.

Tool	What it does
tp_scan_project	Scan codebase: detect framework, extract routes, API endpoints, forms, auth files. Cache saved to .testpilot/context.json. Pass path to the project root.
tp_start	Start a session. Auto-loads cached scan. Pass project_path to your project root.
tp_get_context	Get relevant source files for a feature. Returns actual code for Claude to read before planning.
tp_plan	Generate a step-by-step test plan with real Playwright tool calls, real field names, and screenshot steps built in.
tp_assert	Record a pass/fail assertion. Pass screenshot_file (the filename you gave to browser_take_screenshot) to embed the screenshot in the report.
tp_end	Close the session and write HTML + JSON reports.

Screenshot pattern

When tp_plan generates a browser_take_screenshot step, use this pattern:

# Step from tp_plan:
browser_take_screenshot({ filename: "screenshot-pageload.png" })  ← Playwright MCP

# Then immediately:
tp_assert({ label: "Page loaded", passed: true, screenshot_file: "screenshot-pageload.png" })

testpilot reads the file from /tmp/testpilot-screenshots/screenshot-pageload.png and embeds it in the HTML report.

Supported frameworks

Framework	Routes	API endpoints	Forms
Next.js App Router	✅ app/**/page.tsx	✅ app/api/**/route.ts	✅
Next.js Pages Router	✅ pages/**/*.tsx	✅ pages/api/**	✅
React Router + Vite	✅ via router config	—	✅
Express / Fastify	—	✅ app.get/post/put/delete	✅
Nuxt / SvelteKit	partial	partial	✅
Monorepos (Turborepo / pnpm)	✅ auto-detected	✅	✅

Manual workflow (for more control)

tp_scan_project { path: "/absolute/path/to/your/project" }
tp_start { project_path: "/absolute/path/to/your/project", name: "Sprint 4 — Login" }
tp_get_context { feature: "login" }
tp_plan { feature: "login flow", edge_cases: true }

# Claude executes each step, then:
tp_assert { label: "Login page loaded", passed: true, screenshot_file: "screenshot-pageload.png" }
tp_assert { label: "Form submits successfully", passed: true, screenshot_file: "screenshot-aftersubmit.png" }

tp_end

How AI tools discover this server

When Claude Code or Cursor connects to testpilot-mcp, the server sends a built-in instructions block that explains the full workflow, tool order, and screenshot pattern. You don't need to prompt Claude with setup instructions — it reads the workflow from the server on connection.

Local development

git clone https://github.com/AishwaryShrivastav/testpilot.git
cd testpilot
npm install
npm test   # all 28 integration checks — no browser needed

Point testpilot at the local copy:

"testpilot": {
  "command": "node",
  "args": ["/absolute/path/to/testpilot/src/index.js"]
}

Requirements

• Node.js 18+
• @playwright/mcp — browser driver (Chromium bundled), configured with --output-dir /tmp/testpilot-screenshots
• testpilot itself: zero heavy dependencies (MCP SDK + Zod only, ~2 MB)

Contributing

• Fork → feature branch
• Edit src/
• npm test — must pass all 28 checks
• Pull request

Issues & requests: GitHub Issues

License

MIT