When a problem only appears with several extensions enabled, disabling them one at a time can change loading order, lose a dependency, or mistake a new error for the original bug. This CLI runs an explicit scenario in fresh real Pi processes, reduces the extension set, and verifies the remaining candidates again.

It is a developer CLI, not an extension to load into Pi. It has no runtime npm dependencies. Node.js 22+ and macOS or Linux are required. The E2E suite pins Pi 0.85.1; other Pi versions are not yet verified.

Install

npm install --global pi-extension-repro
pi-extension-repro reduce ./repro.json

Install Pi separately and make pi available on PATH, or point the scenario at a local Pi CLI file as described below.

Try the included conflict

git clone https://github.com/leesama/pi-extension-repro.git
cd pi-extension-repro
npm ci --ignore-scripts
npm run demo

The demo loads three extensions. stamp followed by check triggers REPRO_ORDER_CONFLICT; harmless is unnecessary. It makes no model requests and needs no API key.

The last output is a JSON summary similar to:

{
  "status": "reproduced",
  "selected": ["stamp", "check"],
  "minimality": "1-minimal",
  "trials": 28,
  "report": "/your/checkout/.pi-repro/reduce-TIMESTAMP/report.json"
}

Use the actual report path printed by your run:

node bin/pi-extension-repro.js replay /your/checkout/.pi-repro/reduce-TIMESTAMP/report.json

Reversing stamp and check in the scenario removes this particular failure. Reduction always preserves the supplied order.

To install the CLI from a checkout:

npm install --global . --ignore-scripts
pi-extension-repro --help

Describe your failure

Start with a small fixture version of the problem. Entrypoint files are loaded from their original locations so their normal relative imports and installed dependencies still work. Paths inside the JSON are relative to that JSON file.

{
  "version": 1,
  "pi": { "command": "pi" },
  "extensions": [
    { "id": "driver", "path": "./fixtures/driver.ts", "required": true },
    { "id": "first", "path": "./extensions/first.ts" },
    { "id": "second", "path": "./extensions/second.ts" }
  ],
  "failure": {
    "source": "extension_error",
    "contains": "Conflicting command registration"
  },
  "steps": [],
  "watch": ["./package-lock.json"],
  "repetitions": 3,
  "timeoutMs": 10000,
  "maxTrials": 200
}

pi-extension-repro reduce repro.json --out .pi-repro/registration-bug
pi-extension-repro replay .pi-repro/registration-bug/report.json

For a specific local Pi installation, use "pi": { "cli": "./node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js" } or --pi /path/to/pi-cli.js. --pi paths are relative to the calling directory. A recorded replay uses its original host; it does not accept a different --pi.

Field	Meaning
`extensions`	Ordered local entrypoint files, each with a unique `id` and real path. No package discovery or installation is performed. An empty list is valid for a host-only reproduction.
`required`	Keep this extension in every candidate. Useful for a fixture driver or shared dependency.
`group`	Entries with the same group are removed together. Members can be interleaved; their original order is preserved. If one member is required, the entire group is required.
`failure.source`	`extension_error`, `stderr`, `rpc_error`, `assistant_error`, or `tool_error`.
`failure.contains`	Case-sensitive literal substring, not a regular expression. Pick a distinctive error signature.
`steps`	Ordered prompts. Empty means startup and an RPC `get_state` handshake only.
`watch`	Additional files to fingerprint, such as imported helpers, package manifests and lockfiles.
`repetitions`	2–10 independent processes per candidate, default 2. Mixed outcomes are unresolved.
`timeoutMs`	Total deadline per process, including startup and shutdown, default 10,000 ms.
`maxTrials`	Process budget, default 200. A candidate is started only when its full repetition budget is available.

For a slash command, use {"message":"/my-command args","waitFor":"response"}. The command must appear in Pi's get_commands response; removing its provider makes that candidate unresolved. This waits for the RPC command response. Detached asynchronous work launched by a command is outside this completion boundary. Interactive confirmation/input dialogs make a trial unresolved.

Reproduce a model/tool interaction without a paid model

Configure canned OpenAI-compatible streaming replies. Each trial starts a fresh loopback HTTP server with its own reply queue. Pi still processes a real agent turn and executes the actual extension tool.

{
  "steps": [
    { "message": "Run the diagnostic tool.", "waitFor": "agent_settled" }
  ],
  "model": {
    "replies": [
      { "toolCalls": [{ "name": "diagnose", "arguments": {} }] },
      { "text": "Diagnostic complete." }
    ]
  }
}

Add these fields to your scenario; keep an extension that supplies diagnose marked required if other extensions modify its behavior. A request beyond the reply queue is unresolved. Unused replies are allowed because an extension may intentionally short-circuit a turn. This is deterministic input injection, not an assessment of a real LLM's quality or provider compatibility.

What the result means

Every process is classified before its repetitions are aggregated:

fail: the configured target error was observed, with no different error or infrastructure failure.
pass: all configured steps and shutdown completed successfully without any error.
unresolved: timeout, inconsistent repetitions, different error, failed process startup, abnormal exit without target evidence, interactive dialog, invalid RPC output, or output beyond 1 MiB.

Pi can emit an extension error and still exit with code 0. The runner inspects structured RPC errors instead of equating a successful exit with a passing scenario. Nonempty stderr is treated conservatively as an error; for a stderr target, the literal is matched against that trial's whole stderr stream.

The reducer checks the full set, checks the required-only set, tries ordered subsets and complements, then freshly reruns the remaining set and each possible single-atom removal. An atom is one extension or one configured group.

1-minimal means that, in the final recorded checks, the selected set reproduced and removing any one optional atom passed. This is conditional on the tested scenario, fixed required extensions, group boundaries and observed stability. It is not a globally smallest solution, a proof of causation, or a guarantee against rare nondeterminism. Non-monotonic failures can have smaller solutions that require removing several atoms together.

If a removal is unresolved, the result says minimality: "inconclusive". If the final reproduction changes, inputs change, or the budget runs out, no verified minimality is claimed. An empty selected set means no supplied optional extension is needed for the target.

CLI exit	Meaning
`0`	Target reproduced consistently. Check `minimality` separately.
`1`	Target absent in the baseline or replay.
`2`	Unresolved, interrupted, budget exhausted, fingerprint drift, or invalid input.

Reports and replay

Each output directory contains:

report.json: normalized scenario, selected IDs, Pi/Node/platform versions, SHA-256 file fingerprints, candidate order, per-process verdicts, error digests, exit codes and durations.
scenario.json: the selected extensions with the same scenario settings, usable for a new reduction.

Replay reruns only the selected extensions, checks recorded file fingerprints and host versions, and refuses changed inputs. It validates reproduction, not minimality. An unselected extension file can be removed without preventing replay. Output directories must be new; existing results are never overwritten. Ctrl+C terminates the current trial and writes an interrupted report when the run has already started.

Reports reference original files; they do not bundle source code. Entrypoint files, explicitly supplied Pi CLI files, and watch files are hashed. Imported modules and node_modules are not automatically traversed or snapshotted. Add relevant files to watch, retain your lockfile, and use the same installed dependencies. When using pi.command, its version is recorded but the executable itself is not hashed. Reports use local absolute paths; moving a reproduction to another machine requires adapting the scenario and running a new reduction.

Raw stdout, stderr, model requests and session transcripts are not stored. Error text is reduced to its digest and target-match flag. The scenario itself, including prompts, canned replies, error signature and local paths, is stored for replay. Review these inputs before sharing a report. Files are written with owner-only permissions.

Isolation boundary

Every trial gets a temporary home, agent config, cache and working directory. User provider keys and other environment variables are not inherited, apart from PATH, locale and SystemRoot. Extension discovery, context files, built-in tools, startup networking and telemetry are disabled. Model steps use the local fixture model; the CLI does not pass real provider credentials.

This is process/configuration isolation, not an OS security sandbox. Extensions are trusted executable code. They can access original source directories, open network connections, read explicitly addressed files or spawn detached processes. Ordinary descendants in the trial's process group are terminated on cleanup; a process that creates its own session can escape that group. For untrusted extensions, run the entire CLI in a separate container or VM. Failures that depend on your real project files, global settings, TUI, live providers or background state must first be reduced to an explicit fixture; those inputs are not copied automatically.

Development and E2E

npm ci --ignore-scripts
npm test
npm run test:e2e

The E2E suite packs this project, installs the tarball into a clean temporary prefix, invokes its installed executable and starts real Pi 0.85.1 processes. It covers single-extension failure, interacting extensions, reversed order, required/grouped dependencies, timeout and descendant cleanup, alternating failures, unrelated dependency errors, tool execution against a local model, configuration isolation, report replay, fingerprint drift and budget exhaustion. No provider account is needed.

CI runs the same suite on Linux and macOS with Node 22 and 24.

中文说明

这个工具面向 Pi 扩展作者：把故障写成明确的本地场景，保留扩展加载顺序，多轮启动真实 Pi，缩减出仍能复现问题的扩展组合。

fail 是命中了指定故障，pass 是场景完整执行且没有错误，unresolved 表示超时、其他错误或重复结果不一致。只有最终复现与逐项移除检查都通过时，才标记条件性的 1-minimal；不会把它称为全局最小组合。

快速体验：npm ci --ignore-scripts && npm run demo。报告可用 node bin/pi-extension-repro.js replay <report.json> 重放。扩展仍然是有文件和网络访问能力的本地代码，进程隔离不等于安全沙箱。

Pi extension documentation defines the extension lifecycle.
Pi RPC documentation describes the process protocol used by this runner.
pi-coding-agent-test provides a general real-Pi E2E testing framework. This project focuses on repeated failure classification, ordered input reduction and replay.

MIT licensed.