@testorim/cli

Testorim CLI

Trigger AI browser QA tests from your terminal or CI pipeline.

Monorepo: Repository README · Backend / API: packages/backend

npm install -g @testorim/cli
testorim login
testorim list
testorim run <procedure-id>

Quick start

• Generate an API key at https://app.testorim.com/settings/api-keys.

• Sign in:

  testorim login
  

  The key is saved to ~/.testorim/config.json with 0600 perms. You can also pass it via the TESTORIM_API_KEY env var (preferred for CI).

• Run a saved procedure:

  testorim run <procedure-id>
  

  Exits 0 on pass, 1 on fail, 2 on error, 3 when the run needs review — perfect for CI.

Ad-hoc tests

testorim run \
  --project <project-id> \
  --description "Sign in with [e-mail] and verify the dashboard loads"

Override the project's base URL for a single run (e.g. point at a PR preview):

testorim run <procedure-id> --base-url https://preview-42.staging.example.com

Negative tests

testorim run \
  --project <project-id> \
  --description "Try to log in with the wrong password and verify the error" \
  --expectation fail

The --expectation flag frames the report — fail means the run "passes" when the app correctly rejects.

Use it from a coding agent (MCP)

testorim mcp is a Model Context Protocol server on stdio. Any agent that speaks MCP can then run a Testorim test and read the verdict: Claude Code, Codex, Cursor, VS Code (Copilot), Windsurf, Antigravity, Gemini CLI, Claude Desktop and others.

It gives the agent six tools:

Tool	What it does
list_projects	The projects (sites under test) in your workspace
list_tests	A project's saved tests
run_test	Runs a saved test, or any test described in plain English, and waits for the verdict: which steps failed, whether the app or the test was at fault, the report, and links to the video and trace
get_run	Reads a run's result
cancel_run	Stops a run
create_project	Adds a site to test

Agents start MCP servers without your shell's environment, so put the key in the agent's config. Create a key in Testorim under Settings, API keys.

Claude Code

claude mcp add testorim -e TESTORIM_API_KEY=tst_live_... -- npx -y @testorim/cli mcp

Codex (~/.codex/config.toml)

[mcp_servers.testorim]
command = "npx"
args = ["-y", "@testorim/cli", "mcp"]
env = { TESTORIM_API_KEY = "tst_live_..." }

Cursor (.cursor/mcp.json), Windsurf (~/.codeium/windsurf/mcp_config.json), Antigravity (Manage MCP servers, View raw config), Gemini CLI (~/.gemini/settings.json) and Claude Desktop (claude_desktop_config.json) all take the same block:

{
  "mcpServers": {
    "testorim": {
      "command": "npx",
      "args": ["-y", "@testorim/cli", "mcp"],
      "env": { "TESTORIM_API_KEY": "tst_live_..." }
    }
  }
}

VS Code (.vscode/mcp.json)

{
  "servers": {
    "testorim": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@testorim/cli", "mcp"],
      "env": { "TESTORIM_API_KEY": "tst_live_..." }
    }
  }
}

Then ask the agent something like: "Deploy a preview, then use Testorim to check that a new user can sign up and reach the dashboard."

The site has to be reachable from the internet: Testorim's browsers refuse localhost and private addresses. To test work in progress, run it against a preview deployment (run_test takes a baseUrl) or expose your dev server through a tunnel. Add "TESTORIM_API_URL" to env if your workspace lives on another host.

CI usage (GitHub Actions example)

- name: Run smoke test
  env:
    TESTORIM_API_KEY: ${{ secrets.TESTORIM_API_KEY }}
  run: npx @testorim/cli run <procedure-id>

Flags

Flag	What
--api-key <key>	Override stored key
--api-url <url>	Override API base URL (default https://app.testorim.com)
--project <id>	Project id (required for ad-hoc tests, optional otherwise)
--description "..."	Plain-English test prompt for ad-hoc runs
--base-url <url>	Override the project's base URL for this run only
--expectation <pass|fail|unknown>	Test intent (default pass)
--json	Machine-readable output
--quiet	Suppress progress lines

Exit codes

Code	Meaning
0	Run passed (overallStatus = "passed")
1	Run failed (overallStatus = "failed")
2	Couldn't run (auth error, bad input, server error, timeout)
3	Needs review (overallStatus = "needs_review"): every step passed, but the report could not confirm the request was met

Configuration

Precedence (highest first):

• CLI flags
• Environment variables: TESTORIM_API_KEY, TESTORIM_API_URL
• ~/.testorim/config.json