comber

Comber

Every-click AI QA agent. Point it at a URL and it walks the app like a real user, clicks what's safe, watches for breakage, and writes a self-contained HTML report — leaving cheap deterministic replays behind so the expensive every-click run happens once.

Comber by RADLAB — product site codecomber.io. CLI/package: comber.

What it does today. Runnable DOM-first web crawler with breakage detection, optional Claude action-picker + judge, and an HTML report. Four drivers now sit behind the universal contract — web (Playwright), api (OpenAPI), native (iOS/Android over a pluggable Maestro runner), and desktop (Windows UI Automation or macOS Accessibility). The rest (Stagehand selector-cache replay polish, GitHub Action on Vercel preview deploys) is staged — see Roadmap below.

Why DOM-first

The accessibility/DOM snapshot is ~200–400 tokens per step with stable element refs; a screenshot is ~1,000–1,800 image tokens per step. So Comber drives the browser off the DOM (Playwright today, Stagehand selector-cache next) and reserves Claude vision/computer-use for the few surfaces the DOM can't describe (canvas, file-preview, drag-drop, visual diff). That keeps a full crawl in the cents, deterministic, and CI-safe.

Install

Comber ships as the comber package (CLI + reusable GitHub Action). The published tarball carries a prebuilt dist/ plus action.yml, so no build step is needed on a normal npm/bun install.

# Pinned one-off quickstart (install the matching Chromium once, then crawl):
npx -y comber@0.1.15 install-browsers
npx -y comber@0.1.15 check https://example.com

# Optional project/global installs:
bun add --exact comber@0.1.15
npm i -g comber@0.1.15

# Then run the installed CLI:
comber check https://example.com
comber check https://example.com --max-states=30 --llm

Public docs, the reusable Action, and the connector contract + catalog live at gitlab.com/getcomber/comber-public. Comber itself is proprietary and its source is not public, so npm is the install path — the tarball carries a prebuilt dist/, and pinning an exact version with --exact is the way to hold a build steady.

Run from source

cd ~/repos/comber
bun install
bunx playwright install chromium

# Heuristic crawl (no API key needed):
bun run dev https://example.com

# LLM-judged (Opus judges findings; heuristic action ordering):
cp .env.example .env && echo "ANTHROPIC_API_KEY=sk-ant-..." >> .env
bun run dev https://example.com --max-states=30 --llm

# Authenticated crawl against a SYNTHETIC account:
#   COMBER_STORAGE_STATE=./auth.json bun run dev https://example.com

Generating a synthetic session

An authenticated crawl needs a Playwright storageState (cookies + localStorage) for a synthetic / throwaway account — never a real user's. Point the CLI at one via COMBER_STORAGE_STATE:

COMBER_STORAGE_STATE=auth.json comber https://example.com

Any Playwright storageState JSON works (e.g. one produced by your own Playwright context.storageState({ path })). This repo also ships two capture helpers — run them from a checkout (git clone + bun install), as they live under scripts/ and aren't part of the published npm package:

# Interactive: opens a browser, you log in, press ENTER to capture.
bun run save-session https://example.com/login auth.json

# Headless (CI-friendly): credentials from the environment, never the CLI.
SYNTH_EMAIL=[e-mail] SYNTH_PW=… bun run login https://example.com/login auth.json
#   …or read the password from a file instead of $SYNTH_PW:
SYNTH_EMAIL=[e-mail] bun run login https://example.com/login auth.json ./pw.txt

The saved file holds live session tokens — .gitignore already covers auth*.json / *-auth.json / auth/. Keep it there.

Flags: --max-states=N --max-actions=N --max-ms=N --max-tokens=N --headed --llm --cross-origin --external-links --allow-writes --baseline=PATH --pr-comment=PATH --config=PATH --profile=NAME --strict (exit 1 on any fail-severity finding; default is soft).

API target (comber api …): --openapi=PATH|URL (JSON or YAML) --api-base=URL --graphql=URL --har=PATH --seed=PATH --allow-writes. Credentials are env-only: COMBER_API_TOKEN (single Bearer) and COMBER_API_AUTH_PROFILES (a JSON {role:token} map for the multi-role IDOR/broken-auth harness). See API driver below.

Profiles, baselines, and PR comments

Profiles keep per-app crawl settings out of CI YAML:

cp comber.config.example.json comber.config.json
bun run dev --profile local --config comber.config.json

comber.config.json can be either one direct config object or a named profiles object with defaultProfile. Supported aliases include url/startUrl, maxMs/maxWallClockMs, maxTokens/maxTokenBudget, maxPerState/maxActionsPerState, maxRegotos/ maxRegotosPerState, nearDup/nearDupThreshold, storageState/storageStatePath, baseline/baselinePath, and prComment/prCommentPath.

Baseline diffing compares current findings by stable signature against a prior result.json:

bun run dev https://preview.example.com --baseline baselines/preview-result.json

When no baseline exists, Comber treats all current findings as new and keeps running. To produce a Markdown body for a non-blocking pull-request comment:

bun run dev https://preview.example.com --baseline baselines/preview-result.json --pr-comment runs/pr-comment.md

Informational findings

Comber reports fail, warn, and info severities. Strict gates and baseline headline counts use only fail and warn; info findings are visible context that should not block a run.

The web crawler normalizes crawler-caused noise before dedupe:

• Comber-blocked egress fan-out (egress-blocked plus matching request/console/fetch errors) becomes one crawler-suppressed info finding.
• Known telemetry infrastructure such as Cloudflare Insights, Statsig, Sentry ingest, and Microsoft browser events becomes third-party-telemetry info.
• Turnstile, hCaptcha, and Cloudflare challenge endpoints or widget errors become bot-challenge info findings.
• .ico images are not marked broken-image solely because browser decode reports naturalWidth === 0.

The example workflow .github/workflows/comber.yml runs Comber from source. It uploads runs/ as an artifact and can optionally post runs/pr-comment.md to a PR when pr_number is supplied. For dropping Comber into another repo, use the reusable Action below.

.github/workflows/ holds consumer examples only — copy them into your repo. Comber's own CI runs on Forgejo, mirrored to GitLab; nothing under .github/ gates this repository.

CI integration (reusable Action)

action.yml at the repo root is a composite GitHub Action so any repo can run Comber on a preview deploy. It sets up bun → installs comber → runs it → uploads the runs/ report artifact → posts a non-blocking PR comment → and gates the job only when strict and fail-severity findings are present.

Action status: the action is complete and action.yml ships in the npm tarball today, but the hosted uses: getcomber/comber@v1 listing is not currently available — treat the yaml below as the Action's shape, not a working reference.

Nothing is blocked by that. The CLI is the supported path and runs in every pipeline, GitHub included: install the matching browser with npx -y comber@0.1.15 install-browsers, then run npx -y comber@0.1.15 check <preview-url> --strict. Consumers who want the composite-action ergonomics can vendor action.yml from the tarball and reference it locally with uses: ./.

# .github/workflows/comber-pr.yml in the consumer repo
name: comber
on:
  pull_request:
    types: [opened, synchronize, reopened]
permissions:
  contents: read
  pull-requests: write
jobs:
  comber:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: getcomber/comber@v1 # NOT yet resolvable — see Action status above
        with:
          url: https://your-preview-deploy.example.com
          max-states: "20"
          strict: "false" # non-blocking PR comment first; flip to "true" to fail on defects
          pr-comment: "true"
        env: {}
        # secrets:
        #   anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} # only when llm: "true"

For an API target, swap url for openapi + api-base:

      - uses: getcomber/comber@v1 # NOT yet resolvable — see Action status above
        with:
          openapi: ./openapi.json # file path or http(s) URL
          api-base: https://api-preview.example.com
          strict: "true"

Inputs

Input	Default	Purpose
url	""	Web target URL to crawl (use this or openapi).
openapi	""	OpenAPI 3.x source (path/URL); selects the API driver.
api-base	""	API base URL (required with openapi).
seed	""	API seed-data JSON path (per-operation overrides).
max-states	""	Max states to exercise (blank = Comber default).
llm	"false"	Use the Claude action-picker + judge (needs anthropic-api-key).
external-links	"false"	Check external links (egress to private/metadata hosts stays blocked).
allow-writes	"false"	Allow mutating HTTP methods — synthetic environments only.
strict	"false"	Exit non-zero (fail the job) when fail-severity findings exist.
pr-comment	"true"	Post a non-blocking comment on the PR.
baseline	""	Prior result.json for baseline diffing.
config / profile	""	comber.config.json path / profile name.
install-spec	"comber"	Spec passed to bun add (see the publish gate).
comber-version	""	Version pin when install-spec is the default (→ comber@<version>).
bun-version	"latest"	Bun version for oven-sh/setup-bun.
anthropic-api-key	""	ANTHROPIC_API_KEY pass-through (LLM mode).
storage-state	""	Absolute path to a synthetic Playwright storageState (COMBER_STORAGE_STATE).
api-token	""	Bearer for the API target (COMBER_API_TOKEN).
sink-file / sink-dir / sink-url / sink-token	""	COMBER_SINK_* persistence pass-through.
github-token	${{ github.token }}	Token for gh pr comment (needs pull-requests: write).
pr-number	""	PR to comment on (defaults to the triggering pull_request).

The job's only output, exit-code, is Comber's process code (0 clean/soft · 1 strict + fails · 2 crash).

Script-injection safety

Every consumer-supplied value (url, openapi, api-base, …) is forwarded to the run step through env: and read inside a quoted bash array as "$VAR" — it is never interpolated into a run: script via ${{ … }}. That is the GitHub Actions untrusted-input rule: a url like "; rm -rf / # cannot break out of an env var into the shell. The action follows the same pattern as .github/workflows/comber.yml, and pins every third-party action to a commit SHA.

Synthetic session + secrets

An authenticated crawl needs a Playwright storageState for a synthetic / throwaway account (never a real user). Generate it once (see Generating a synthetic session above), store it as a CI secret, write it to a file in a prior step, and pass its absolute path:

      - name: Materialize synthetic session
        run: printf '%s' "$STATE" > "$GITHUB_WORKSPACE/auth.json"
        env: { STATE: ${{ secrets.COMBER_STORAGE_STATE }} }
      - uses: getcomber/comber@v1 # NOT yet resolvable — see Action status above
        with:
          url: https://your-preview-deploy.example.com
          storage-state: ${{ github.workspace }}/auth.json
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
          llm: "true"

Secrets are pass-through only — they reach Comber as env (ANTHROPIC_API_KEY, COMBER_STORAGE_STATE, COMBER_API_TOKEN, COMBER_SINK_*) and never appear in a run: script, the report, the PR comment, or logs. Every text artifact (report, result.json, plan.json, desktop automation trees, PR comments, sink records, and logs) is redacted before it leaves the process. Per-state screenshots (state-*.png) are raw pixels of the target and can carry on-screen secrets or PII that text redaction cannot reach, so screenshot capture is off by default. Enable it only with --screenshots or screenshots: true in a profile, and keep the resulting run directory access-restricted with the shortest practical retention. See docs/DATA_HANDLING.md.

Releases

comber is live on npm — bun add comber / npm i -g comber resolves the published package (prebuilt dist/ + action.yml in the tarball). The Action defaults to install-spec: comber; pin a version with comber-version: <ver> (→ bun add comber@<ver>), or point install-spec at a git ref to run an unreleased branch — the prepare script builds dist/ on install either way.

New releases are prepared from the authoritative Forgejo repository and may be initiated only by a single designated human release authority. Releasing is split across two planes, deliberately:

• Forgejo gates. A manual, fail-closed workflow verifies the exact tag, commit, changelog, package artifact, clean install and secret scans, packs the exact bytes and scans them, writes a machine-readable release record, and refuses a version the registry already serves. It holds no publish credential and does not publish.
• The GitLab mirror publishes. On a semver tag only, via npm Trusted Publishing — a short-lived OIDC token rather than a stored secret.

So there is no long-lived npm credential anywhere in the release path.

Comber does not carry an npm provenance attestation, and cannot: npm generates one only when the publishing project is public and matches package.json's repository, and Comber is proprietary — its source is private. Trusted Publishing and provenance are easy to conflate, so to be exact: the first is how the publish authenticates, the second is a signed statement binding an artifact to a public source commit. Comber has the first and not the second.

What you can verify:

npm view comber@<ver> version dist.integrity   # matches the tarball you installed
npm audit signatures                            # npm's registry signature over that tarball

The full procedure and the per-version release records live in the source repository. AI agents do not tag, dispatch, publish, deploy, approve, or merge releases.

API driver — OpenAPI / GraphQL / HAR + multi-role

The API target discovers an API's operation set, exercises what's safe, and classifies breakage into its own signal set — the same discover → exercise → classify → dedupe → report contract as web/native. It read-only by default (only GET/HEAD/OPTIONS for REST, queries for GraphQL, the recorded reads for HAR); writes/mutations are surfaced as a counted coverage skip and never fired unless --allow-writes. Four spec sources feed the same pipeline:

comber api --openapi ./openapi.json --api-base https://api-preview.example.com   # OpenAPI (JSON)
comber api --openapi ./openapi.yaml --api-base https://api-preview.example.com   # OpenAPI (YAML; sniffed)
comber api --graphql https://api-preview.example.com/graphql                     # GraphQL introspection
comber api --har ./recorded-traffic.har                                          # recorded HAR capture

Source	Discover	Exercise	Breakage signal(s)
OpenAPI (JSON/YAML)	parse paths → operations ($refs resolved)	build a concrete request per operation	http-5xx, undocumented-status, content-type-mismatch, response-schema-violation (full ajv JSON-Schema validation, carrying the offending paths), response-shape-mismatch (light fallback), broken-auth, broken-object-authorization
GraphQL	send the standard introspection query → query/mutation fields	POST a minimal valid operation document	graphql-error (non-empty errors[]), graphql-http-status (non-200), request-failed
HAR	recorded request entries, deduped by route template	re-send and diff against the recorded response	contract-drift (status or body-shape divergence), request-failed

Response-schema validation (ajv). OpenAPI 2xx bodies are validated against the operation's declared response schema with ajv + ajv-formats ($refs resolved, cycles broken, OpenAPI nullable honored). A violation emits response-schema-violation carrying only the ajv error paths (/items/0/id must be integer) — never the body's values, so it stays redaction-safe. When no compilable schema is declared, the light required-property check (response-shape-mismatch) is the fallback.

Multi-role auth → broken-auth + IDOR. Supply named credential sets (role → token) via config or $COMBER_API_AUTH_PROFILES (a JSON { "role": "token" } map — never a CLI flag, env wins over config). With ≥ 2 roles plus seed data tying a resource to its owning role, operations marked ownerScoped get a cross-role object-access check: Comber confirms role B can read B's resource, then re-issues the same request with role A's token — a 2xx for A is broken-object-authorization (a classic IDOR). The no-credential broken-auth check (auth-required op returns 2xx without a credential) is unchanged. Opt-in + sandbox-only.

// comber.config.json (api profile)
{
  "target": "api",
  "openapi": "./openapi.json",
  "apiBase": "https://api-sandbox.example.com",
  "apiAuthProfiles": { "alice": "", "bob": "" }, // tokens via $COMBER_API_AUTH_PROFILES
  "ownerScoped": ["GET /users/{id}"],
  "roleSeed": "./role-seed.json"                  // { "alice": { "GET /users/{id}": { "path": { "id": 1 } } }, "bob": { … } }
}

Deferred (honest): the IDOR harness checks operator-declared ownerScoped operations against operator-supplied per-role seed data. Full automatic resource-ownership discovery — creating a resource under role B at runtime and tracking its id across operations to test every read path — is deferred; today the operator declares which operations are owner-scoped and which resource each role owns. GraphQL argument synthesis covers scalars/enums/required-input-objects with minimal literals; exotic custom scalars fall back to a string sample.

Native (mobile) driver — Maestro

Comber's third target is a native iOS/Android app, driven over Maestro. It proves the Driver abstraction holds for a non-page, non-API surface: the core's frontier / dedupe / caps / report / baseline / redact / triage / coverage run unchanged against the screens, controls, and breakage signals the native driver emits.

# Android app already installed on a booted emulator (default platform = android):
comber native --app com.example.app

# iOS bundle on a booted simulator, a specific device, and declared deeplinks:
comber native --app com.example.app --platform ios --device "iPhone 15" \
  --deeplink "myapp://settings" --deeplink "myapp://profile"

Flags: --app <appId|apk|path> (required) · --platform android|ios (default android) · --device <id> · --deeplink <uri> (repeatable). The same --profile / --config / --strict machinery applies; a profile can set target: "native", app, platform, device, deeplinks.

Evidence flags for native/OEM runs: --provider <name>, --device-class phone|tablet|foldable|emulator|simulator|unknown, --device-model <model>, --os-version <version>, --android-version <version>, --app-version <version>, --samsung, --one-ui <version>, and --build-fingerprint <fingerprint>. Evidence artifact flags are repeatable: --evidence-screenshot <path>, --evidence-video <path>, --evidence-log <path>, and --evidence-crash-log <path>. Provider audit fields include --provider-run-id <id> and --verification-command <command>. These populate result.json.extra.deviceEvidence.

Comber validates native evidence at runtime. Blank artifact paths, unknown artifact kinds, provider run ids without providers, and incomplete verified proxy evidence are surfaced in result.json.extra.deviceEvidence.validationErrors and block exhaustive claims through the typed native-evidence-invalid blind spot. Every native run also includes result.json.extra.nativeReadiness, a provider-neutral preflight summary for device id, provider, verification command, evidence artifacts, and Samsung/OEM claim readiness.

Native network egress remains a blocker unless the run declares complete verified proxy evidence: --egress-proxy-verified, --egress-proxy-provider <name>, --egress-proxy-command <command>, and --egress-proxy-artifact <path>. All three support fields are required; an incomplete proxy claim is still reported as native-egress-unenforced.

Mac/iOS runs can produce that evidence through the bundled provider-neutral proof helper. It does not install or configure a proxy service; it runs an operator-supplied local proof command with execFile argv semantics, requires expected output markers, writes a JSON proof artifact, and prints the --egress-proxy-proof <path> flag for the native run:

bunx tsx scripts/native-ios-proxy-verify.ts \
  --device CAE50036-EF58-4880-B368-67CBD2EF0728 \
  --provider local-mac-proxy \
  --proxy-command "mitmproxy/Proxyman profile active for simulator CAE50036-EF58-4880-B368-67CBD2EF0728" \
  --proof-command-json '["xcrun","simctl","spawn","CAE50036-EF58-4880-B368-67CBD2EF0728","log","show","--last","2m"]' \
  --expect-output comber-proxy-marker \
  --artifact runs/native/ios-proxy-proof.json

comber native --app com.example.ios --platform ios \
  --device CAE50036-EF58-4880-B368-67CBD2EF0728 \
  --egress-proxy-proof runs/native/ios-proxy-proof.json

The proof command must be the real local check that demonstrates the app/device traffic is flowing through the device-level proxy. A host-only curl --proxy ... can show that a proxy listener works, but by itself it is not proof that an iOS simulator or device is routed through that proxy. If the proof command fails, the expected marker is absent, the proof file is missing, or the proof file is malformed, Comber keeps egressProxy.verified: false and continues to report native-egress-unenforced.

Desktop drivers — Windows UIA and macOS Accessibility

Comber's desktop target ships adapter-owned native desktop runners behind the same Driver contract as web, API, and mobile. The Windows path uses UI Automation and replays discovered branch paths through uiapath: addresses. The macOS path uses Accessibility and replays discovered branch paths through axpath: addresses. Neither path changes core traversal semantics.

# Windows: launch an app by executable path:
comber desktop --platform windows --app-path "C:\Apps\Example\Example.exe"

# Windows: attach to an already-running process:
comber desktop --platform windows --process-name Example.exe

# Prefer PID/window-title disambiguation before live action runs:
comber desktop --platform windows --process-id 4242 --window-title "Example"

# Optional local smoke, outside default CI, that performs real safe fixture actions:
bun run smoke:windows-uia -- --fixture --max-states 4 --max-actions 12

# Generic dry-run smoke against an app you provide:
bun run smoke:windows-uia -- --app-path "C:\Windows\System32\notepad.exe" --max-states 1 --dry-run

# macOS: launch by .app path or bundle id:
comber desktop --platform macos --app-path /Applications/Example.app --process-name Example \
  --automation-backend accessibility \
  --provider local-mac \
  --verification-command "osascript -l JavaScript runtime/macos-accessibility-runner.js snapshot" \
  --accessibility-permission-verified

comber desktop --platform macos --bundle-id com.example.Example --process-name Example \
  --automation-backend accessibility

# macOS controlled dry-run smoke. Without --app-path it creates a temporary fixture app.
bun run smoke:macos-accessibility

# macOS desktop proxy proof from already-configured system proxy settings.
# This inspects networksetup state and does not modify macOS proxy configuration.
bun run verify:desktop-macos-proxy -- \
  --app-identity Example \
  --artifact runs/desktop/macos-proxy-proof.json

comber desktop --platform macos --app-path /Applications/Example.app --process-name Example \
  --egress-proxy-proof runs/desktop/macos-proxy-proof.json

Windows controls are identified by AutomationId first, then Name plus tree path, then RuntimeId/path. Buttons, menu items, hyperlinks, tabs, list/data rows, and tree items are treated as branch-capable controls; edits, toggles, radio buttons, combo boxes, and sliders are exercised in place with synthetic input only. Custom, owner-drawn, canvas, DirectX, or otherwise opaque UIA subtrees are reported as desktop-opaque-subtree blind spots.

macOS controls are identified by Accessibility identifier first, then title plus tree path, then role/path. AX buttons, links, tabs, rows, menu items, and tree items are branch-capable; text fields, text areas, toggles, radio buttons, pop-up buttons, and sliders are exercised in place with synthetic input only. Custom, canvas, Metal, WebView, and otherwise opaque Accessibility regions are reported as desktop-opaque-subtree blind spots. If the terminal or Codex host process lacks macOS Accessibility permission, the runner fails closed and Comber reports desktop-accessibility-unavailable.

Desktop evidence lands in result.json.extra.desktopEvidence; readiness lands in result.json.extra.desktopReadiness. Windows readiness requires app identity, the uia backend, provider/source metadata for complete proof, launch command, verification command, and UIA automation-tree evidence. macOS readiness requires app identity, the accessibility backend, provider/source metadata, launch command, verification command, verified Accessibility permission, and an accessibility-tree artifact. Desktop network egress is still reported as desktop-egress-unenforced unless verified OS-level proxy evidence includes provider, command, and artifact path. bun run verify:desktop-macos-proxy can create that proof only for an already configured macOS system proxy; if no active proxy is found, it writes an unverified artifact and the desktop egress blocker remains. The macOS XCTest backend is reserved in the evidence contract but is not a live default runner yet.

Process-name-only attachment is allowed for inspection, but it can be ambiguous when multiple windows share the same executable. For non-dry-run desktop checks, prefer --process-id and --window-title; the Windows helper refuses missing or ambiguous declared targets instead of falling back to an arbitrary desktop window.

Exhaustive action-tree traversal

Comber treats every discovered surface as an action tree:

• exercise: safe in-place controls are clicked, tapped, typed into, or nudged.
• follow: known destinations such as safe web links are enqueued without clicking through.
• branch: route-less controls such as tabs, accordions, dialogs, menus, SPA buttons, Android/iOS tabs, native links, list cells, Windows UIA branch controls, and macOS AX branch controls are activated; any child state they reveal is added to the frontier.
• probe: a web control whose click would LEAVE the origin — a sign-in button above all — is never allowed to complete. The intended destination is captured at the request gate and the click aborted, then that destination is validated with an anonymous GET that stops at the first off-origin response (a configured auth provider answers with a redirect; an unconfigured one answers 4xx). A hard-dead destination is reported as auth-control-dead / offsite-control-dead; a sign-in control that does nothing at all is auth-control-inert. This closes the classic gap where the one control every customer must use — sign-in — was the one control a safety-correct crawler never verified. The probe proves a provider is configured, never that the account behind it is in good standing; only a real credentialed sign-in proves that, and a crawler must not do it. The same guard covers the FOLLOW path: a same-origin address that answers with an off-origin redirect (an /sso/<provider> redirector) is captured-and-aborted rather than followed onto the third-party page, and the link checker still validates the redirector's chain.

A clean report is only an exhaustive claim when the frontier drains without caps and without unreported blind spots. Capped runs, native/desktop egress limitations, desktop opaque UIA subtrees, closed shadow roots, cross-origin iframes, unavailable devices, and skipped destructive controls are reported as gaps.

Samsung support is Android-plus-OEM validation. Future Samsung profiles must record the exact Galaxy model, Android version, One UI version, provider or device source, command, screenshots/video, logcat crash buffer, and unavailable device classes. A generic Android emulator is not Samsung coverage.

Every run also writes an exhaustivenessLedger into result.json. It records each discovered unit as exercised, followed, branched, skipped, blocked, or blind-spot, with the reason and child-state count when applicable. This is the machine-readable audit trail behind the headline coverage percentage.

For machines with a working Playwright browser install, bun run smoke:web-branch runs a local route-less web branch fixture and verifies from the exhaustiveness ledger that Comber creates a replayable webaction: address and exercises the child control it reveals.

How a phone maps onto the universal contract

Contract	Native meaning
Surface	one screen (the current accessibility/view hierarchy)
SurfaceNode	one control - kind = control type (button/text-input/toggle/tab/link/cell), id = the stable test id (Android resource-id / iOS accessibilityIdentifier / testTag, else text+path), label = visible text, handle = the raw native node, disabled, intent (exercise/follow/branch)
Address	screen identity — id is the goto handle (launch · deeplink:<uri> · tappath:<base>::<tap>…), scope = screen name/route, dedupeKey = destination-screen fingerprint for a tap-path. Frontier de-dups by the goto handle; structural "seen this screen?" is the core's stateFingerprint — two distinct layers
entryAddress()	the launch screen (relaunch, no deeplink)
goto(addr)	relaunch + optional deeplink, or relaunch-base + replay-taps (tap-path)
snapshot()	viewHierarchy() → Surface
exercise(node)	tap (default) or input (text fields); a tap that reaches a new screen returns it as ExerciseResult.discovered (frontier expansion from acting)
probes	session (after-goto) · dom (after-snapshot) · crash (after-exercise) · logs (finalize) — each registered with its lifecycle slot

Native breakage signals: crash (fail — app left the foreground / a FATAL EXCEPTION after an action or launch), anr (warn — an "Application Not Responding" log line), frozen (warn — a tap or viewHierarchy() that times out), error-text (warn — an on-screen error/canary string), assertion-fail (warn — a declared Maestro flow assertion). All dedupe on a signature keyed by screen scope + kind.

The pluggable MaestroRunner transport

The driver depends only on a MaestroRunner interface — the exact same pattern as the API driver's pluggable ApiTransport — so the whole driver is tested hermetically by injecting a stub (no device, no maestro binary). The seam:

interface MaestroRunner {
  launch(appId, deeplink?): Promise<void>;      // the goto reset point
  viewHierarchy(): Promise<NativeView>;          // the current screen's view tree
  tap(selector): Promise<void>;
  input(selector, text): Promise<void>;
  back(): Promise<void>;                          // softReset — dismiss a dialog / pop a screen
  foregroundApp(): Promise<string>;              // crash detection (≠ appId ⇒ died)
  logsSince(sinceMs): Promise<string[]>;         // crash / ANR log scan
  runFlow?(yaml): Promise<void>;                 // optional — declared-flow assertions
  screenshot?(): Promise<Buffer>;                // optional — report evidence
}

The bundled DefaultMaestroRunner shells out to maestro (ephemeral one-command flows for launch/tap/input/back, maestro hierarchy for the view tree) and to adb logcat / dumpsys for crash/ANR detection on Android.

What a LIVE run needs (and what was verified here)

The driver and its mapping/signal logic are covered by a hermetic test suite that stubs the MaestroRunner — no live device crawl was run (none was available). A real run requires, none of which ship with Comber:

• Maestro CLI on PATH (maestro), plus adb for Android crash/ANR logs.
• A booted emulator / simulator (or a Maestro Cloud session) with the app installed.
• The exact DefaultMaestroRunner command strings reconciled against your installed maestro version — they are a best-effort starting point, marked in src/native/maestro.ts, not a CI-verified contract.
• For iOS egress proof, a configured device-level proxy plus a proof command that produces a durable artifact through scripts/native-ios-proxy-verify.ts.

Known gaps (honest, not silent)

• Network-egress safety is NOT enforced on native v1. The web driver's SSRF / write-egress gate works by intercepting browser requests; Maestro cannot proxy traffic, so there is no equivalent wire-level guard here. The only write protection is the name-denylist (destructive controls are observed, never tapped — the core's deniedName over the control label). The egress-hardening path is a device-level proxy (e.g. mitmproxy) the runner routes through. Comber clears the native egress blocker only when complete verified proxy evidence is provided with provider, command, and artifact path, or when a verified --egress-proxy-proof artifact provides those fields. The CLI banner and nativeAudit.egressEnforced: false still surface the Maestro transport limitation on every run.
• Network-egress safety is NOT enforced on desktop v1. Windows UI Automation drives the UI tree; it does not intercept process network traffic. Complete verified OS-level proxy evidence is still required to clear desktop-egress-unenforced.

Abstraction limits the native driver surfaced — RESOLVED in contract v2

Building the third driver stress-tested the fixed Driver contract. Three seams that web + API never exercised showed their edges. Rather than bend the core per-target, the contract was evolved (v2) so all three drivers conform and web + API stay byte-identical (same findings, coverage, dedupe, ordering; the full suite stays green):

• Address.id conflated "dedupe key" with "goto handle" — resolved. For web a URL is both; for API an operation is both. A native screen's structural identity (for dedupe) and its re-reachability (for goto) are different things. The contract now states the two distinct dedup layers explicitly: FRONTIER de-dup is by the goto handle (Address.dedupeKey ?? id, the queued set), while "have I processed this screen?" is the core's stateFingerprint over the Surface (the visited set). id is documented as the goto handle; the optional dedupeKey lets a driver whose handle ≠ structural identity (native tap-paths) collapse distinct handles that reach the same screen. Web/API leave dedupeKey unset → unchanged.
• DriverCapabilities.probes was a closed phase set the orchestrator hard-coded (session/dom/axe/links/forms) — resolved. Probes are now driver-registered with a lifecycle slot ({ name, slot }); the orchestrator iterates each driver's probes at the matching slot. The slot set is fixed and reproduces web's exact order — after-goto (session), after-snapshot (axe, dom, links), after-exercise (forms), finalize — but which probes run is open, so native now registers its own crash (after-exercise) + logs (finalize) phases the closed vocabulary forbade.
• The frontier grew only by FOLLOWING links (outboundAddresses), stranding screens reachable only by tapping — resolved. exercise() may now return ExerciseResult.discovered: Addresses a tap/click navigated to. The orchestrator enqueues them (inScope + caps/dedup), so a native tap-only screen is crawled — returned as a replayable tap-path Address whose goto relaunches the base anchor and re-taps the recorded sequence. Web/API navigate via links (followed, never clicked) and return none. Declaring --deeplink is now an optional seed for depth, no longer the only way past the launch screen.

Sinks / Persistence

By default a run writes only the local HTML report + result.json under ./runs/. Opt-in sinks persist or forward each run's results to any backend — the substrate a hosted tier consumes. The baseline ships two generic, dependency-light adapters (no new dependencies; both use only the Node stdlib). Each is configured purely by environment variables, fans out via a MultiSink, never breaks a run (a sink failure is caught + logged, the run still passes), and is a no-op when unconfigured (nothing set = unchanged local-only behavior).

Every record a sink emits is redacted (redactResult) before it leaves the process, so a crawled target's secrets — finding URLs/messages, signed-URL params, OAuth fragments — never reach a file or cross the wire.

Env var	Sink	Effect
COMBER_SINK_FILE	FileSink	Append one redacted run-summary record (a JSON line) per run to this JSONL file.
COMBER_SINK_DIR	FileSink	Copy the run's screenshots into this directory, under a per-run runId subfolder.
COMBER_SINK_URL	HttpSink	POST the redacted run record (JSON) to this webhook URL. Covers backend ingest + Sentry-style error-tracking collectors.
COMBER_SINK_TOKEN	HttpSink	Optional Bearer token for the webhook — sent as an Authorization header only, never logged and never in the body.

# Local persistence: a growing JSONL log + a screenshot archive.
COMBER_SINK_FILE=./comber-runs.jsonl COMBER_SINK_DIR=./comber-shots bun run dev https://example.com

# Forward each run to a webhook / collector with auth.
COMBER_SINK_URL=https://hooks.example.com/comber COMBER_SINK_TOKEN=… bun run dev https://example.com

Configured sink paths are path-traversal-guarded, and every copied screenshot filename is constrained to stay within the target directory.

Vendor neutrality. The baseline carries only these generic adapters. A vendor-specific backend (a database, object store, or hosted error tracker) belongs in an overlay as its own Sink implementation against the same interface — it is never added to src/ here.

Connectors & marketplace

Where a sink persists the raw record, a connector delivers a finished run to where developers already work — Slack, Discord, a GitHub PR check, a Linear issue, a JUnit report, or any webhook. Connectors are opt-in, declared in the connectors: [...] array of comber.config.json, and each consumes the same redacted RunRecord a sink does — a third-party connector physically cannot receive a crawled-app secret. Six connectors ship built-in; the marketplace is any npm package that satisfies the contract, loaded by module specifier.

{
  "url": "https://preview.example.com",
  "connectors": [
    { "use": "slack", "on": "fail", "mentionOnFail": "<!here>" },
    { "use": "junit", "outFile": "runs/comber-junit.xml" },
    { "use": "@acme/comber-teams", "on": "warn" }
  ]
}

use names a built-in (slack, discord, github, junit, linear, webhook) or an installed package; on gates delivery by severity (always | warn | fail); all other keys are non-secret settings. Secrets come from the environment, never from config (e.g. COMBER_SLACK_WEBHOOK_URL). List the built-ins with comber connectors. Full guide — the built-in table, the RunRecord shape, authoring an external connector, and publishing to the catalog — in docs/CONNECTORS.md.

Architecture

cli.ts → crawl.ts ─ drives ─▶ Driver (driver.ts: WebDriver over Playwright)
            │                     │
            │  orderForExhaustion └─ breakage.ts (CDP listeners, console canaries, axe, signatures)
            │  (heuristic, DOM-first)
            ▼
        triage.ts (Opus "genuine defect?" judge) ──▶ report.ts (self-contained HTML)

• Driver adapter (core/contract.ts) is the seam that makes a surface pluggable. WebDriver (Playwright), ApiDriver (OpenAPI), and NativeDriver (native iOS/Android over a pluggable MaestroRunner) all implement the same shape — so crawl/breakage/report never change.
• Breakage (breakage.ts): deterministic auto-fail (uncaught exceptions, 5xx, failed assets, Next.js regression canaries) + judged tier (axe-core criticals, console noise). Every signal carries a normalized signature so a broken shared footer across 50 pages collapses to one finding.
• Signal normalization (signal-normalize.ts): removes crawler-caused egress fan-out from warn/fail gates while keeping a transparent info finding in the report.
• Frontier crawl (crawl.ts): BFS over same-origin states with fingerprint dedupe and hard caps (states / actions / wall-clock / token budget) — the runaway-cost backstops.

Safety rails (non-negotiable)

• Destructive-action denylist (config.ts): elements named delete/remove/pay/checkout/ cancel/etc. are observed, never clicked.
• Write blocking is on by default: mutating HTTP verbs are aborted unless --allow-writes is passed for a reviewed synthetic environment.
• Private-network egress is always blocked: browser requests and optional external link checks cannot hit localhost, private IP ranges, link-local metadata, or hosts resolving there.
• External link checking is opt-in via --external-links.
• Synthetic accounts only via COMBER_STORAGE_STATE — never point an authed crawl at real user data. Use a dedicated throwaway/test account (e.g. [e-mail]).
• Hard caps abort runaway loops; same-origin confinement by default.
• (Planned) per-app convert-flow guard: upload a 1 KB dummy and STOP before the GPU worker.

Roadmap

Comber's through-line is one contract — discover a target's surface → exercise it → classify breakage → dedupe → report / baseline / gate — so new targets land as new drivers behind it, not core rewrites.

• Now — web MVP: DOM-first crawler, breakage detection, optional Claude action-picker + judge, self-contained HTML report, baseline diffing, PR comment. ✅
• Next — deterministic selector-cache replay (run the expensive every-click pass once, cheap replays after) ✅; generic persistence sinks (JSONL + screenshot dir, webhook/collector POST) ✅; CI integration on preview deploys — reusable composite GitHub Action (action.yml), non-blocking PR comment first ✅ (gated on npm publish — see CI integration above).
• Native — a NativeDriver driving native iOS/Android over a pluggable MaestroRunner behind the same contract ✅ (hermetic; a live run needs maestro + a booted device — see Native driver above).
• Desktop native — Windows UI Automation and macOS Accessibility now ship as desktop adapters behind the same contract ✅. macOS XCTest remains a reserved optional backend, not a default live runner. Desktop evidence/readiness is in place for platform, app identity, automation backend, provider/source, launch command, verification command, screenshots/video/logs/crash logs, accessibility-tree or automation-tree artifacts, and verified OS-level proxy evidence when desktop network egress is claimed controlled.
• API / bot — request/response contract checks as their own breakage-signal set, same pipeline ✅.
• Hosted — multi-tenant dashboard, bring-your-own-key, and billing — the path toward marketplace integrations with CI/deploy platforms.