callme-cli

Callme CLI

Callme CLI is a professional command-line runner for API collections. It runs exported collections outside the desktop UI, loads exported environment variables, executes requests sequentially, runs scripts and contract checks through callman-core, prints terminal results, generates reports, and returns CI-friendly exit codes.

The CLI is intentionally thin: it handles file loading, orchestration, output, reports, and exit codes. Core execution logic stays inside callman-core.

Who This Is For

Business users can use Callme CLI to prove that a business flow still works outside the desktop application, generate shareable HTML reports, and see a clear pass/fail summary.

QA engineers can run regression collections, validate response contracts, inspect failed script tests, and stop early with --bail when a flow is broken.

Developers can debug request execution with --verbose, inspect resolved headers/body/query values, verify environment mutations, and compare CLI behavior with the desktop app.

DevOps engineers can run collections in CI/CD pipelines, enforce exit codes, generate JSON/HTML artifacts, and use --no-fail when a pipeline stage should collect evidence without blocking deployment.

Features

• Runs API collection JSON files.
• Loads environment JSON files exported from the desktop app.
• Resolves environment variables through callman-core.
• Executes requests sequentially.
• Runs pm.* test scripts through core.
• Runs contract validation through core.
• Supports request headers, query params, request body, response body, and response headers in reports.
• Generates JSON and HTML reports.
• Supports CI/CD exit codes.
• Supports advanced execution flags: --no-fail, --bail, --verbose, --silent.

Project Structure

callme_cli/
  src/
    index.ts       CLI command registration and exit handling
    execution.ts   CLI orchestration loop around callman-core
    loaders.ts     Collection and environment file loading/normalization
    format.ts      Terminal output formatting
    reports.ts     JSON and HTML report generation
    summary.ts     Failure detection and summary helpers
    types.ts       CLI TypeScript types
  data/
    register-collection.json
    environment.json
  dist/            Compiled output

Requirements

• Node.js 18 or newer is recommended.
• callman-core must be available. In this workspace it is linked locally with:

"callman-core": "file:../callman_core"

Install And Build

From the callme_cli directory:

npm install
npm run build

Run locally without installing the global command:

npm run start -- run ./data/register-collection.json --env ./data/environment.json

After global linking, the command is:

callme run ./data/register-collection.json --env ./data/environment.json

If global linking fails with a permission error on macOS, use npm run start -- ... during development or configure npm to use a user-owned global prefix.

Basic Command

Callme CLI supports two run modes:

File mode (legacy)

callme run <collection.json> --env <environment.json>

Reads JSON files exported from the desktop app off the local disk. Useful for ad-hoc local runs.

Remote mode (recommended for CI/CD)

callme run \
  --collection-id <collectionId> \
  --environment-id <environmentId> \
  --workspace <workspaceId> \
  --token <cm_pat_...>

CLI fetches the collection + environment from the backend at runtime — no JSON files committed to your repo. The freshest version always runs.

Workspace + collection + environment IDs are visible in the desktop app:

• Workspace ID → Workspace Settings → General → Copy.
• Collection ID → right-side menu next to each collection → Copy ID.
• Environment ID → open the environment → header → Copy.

Personal Access Tokens (PATs) are created in the desktop app: Workspace Settings → CI Tokens → New CI Token. The plain token is shown ONCE on creation — copy it immediately into your CI secret store. Tokens are workspace-scoped and revocable from the same UI.

Configuration sources (remote mode)

Each value falls back through this chain (high → low):

Value	CLI flag	Env var	Default
API URL	--api-url	CALLME_API_URL	https://api.callman.io
Token	--token	CALLME_TOKEN	(required)
Workspace ID	--workspace	CALLME_WORKSPACE_ID	(required)
Collection ID	--collection-id	none	(required)
Environment ID	--environment-id	none	(required)

Cloud user

Default URL works — set 3 secrets (CALLME_TOKEN, CALLME_WORKSPACE_ID) plus the two IDs as command args.

On-prem user

Point the CLI at your internal backend:

export CALLME_API_URL=https://api.callman.acme.local
export CALLME_TOKEN=cm_pat_...
export CALLME_WORKSPACE_ID=...

callme run \
  --collection-id <collectionId> \
  --environment-id <environmentId>

A PAT minted on one backend only works against that backend; cloud PATs do not authenticate against on-prem and vice versa.

Equivalent local development commands

# File mode
npm run start -- run <collection.json> --env <environment.json>

# Remote mode (against local backend)
npm run start -- run \
  --api-url http://localhost:4000 \
  --token <cm_pat_...> \
  --workspace <workspaceId> \
  --collection-id <collectionId> \
  --environment-id <environmentId>

Command Reference

callme run <collection> [options]

Arguments:

Argument	Required	Meaning
<collection>	Yes	Path to a collection JSON file.

Options:

Option	Value	Meaning
--env <envFile>	File path	Loads environment variables from an exported environment JSON file.
--report <json|html>	json or html	Generates a report in the selected format.
--output <file>	File path	Output path for the report. Required when --report is used.
--no-fail	Boolean	Always exits with code 0 after a completed run, even if requests, scripts, or contracts fail. Failures are still printed and included in reports.
--bail	Boolean	Stops execution after the first failed request. Summary and reports include only executed requests.
--verbose	Boolean	Prints detailed debug information for each executed request.
--silent	Boolean	Prints only request status lines and summary. Overrides --verbose.

Input Files

Collection File

The CLI accepts the exported collection shape used by the desktop app and normalizes it before passing requests to callman-core.

Minimum normalized shape:

{
  "name": "Registration",
  "requests": [
    {
      "id": "request-1",
      "name": "Login",
      "method": "POST",
      "url": "https://api.example.com/login",
      "headers": {
        "Content-Type": "application/json"
      },
      "query": {
        "source": "cli"
      },
      "body": {
        "username": "{{env.username}}",
        "password": "{{env.password}}"
      },
      "tests": "pm.test(\"status is 200\", () => { pm.response.to.have.status(200); });",
      "contracts": [
        {
          "path": "data.token",
          "rules": {
            "required": true,
            "type": "string",
            "notEmpty": true
          }
        }
      ]
    }
  ]
}

The CLI also understands desktop export fields such as:

• collection.name
• headers as key/value arrays
• query as key/value arrays
• auth
• testsScript
• postResponseScript
• contractRules

Environment File

Expected environment format:

{
  "name": "staging",
  "variables": {
    "base_url": "https://api.example.com",
    "AccessToken": "token-value"
  }
}

Only variables are passed to callman-core.

Execution Flow

For each request, the CLI:

• Takes an environment snapshot before the request.
• Calls callman-core to run the request.
• Receives response, script results, contract results, and updated environment.
• Takes an environment snapshot after the request.
• Prints output based on selected flags.
• Stops early if --bail is enabled and the request failed.
• Builds a summary from executed requests.
• Writes reports if requested.
• Returns the correct process exit code.

Failure Rules

A request is considered failed if any of these are true:

Failure Type	Rule
Request failure	HTTP status is 0 or >= 400. Status 0 usually means network/request execution failure.
Script failure	Any script test result has status FAIL.
Contract failure	Any contract check result has status FAIL.

This means a request can have HTTP 200 and still fail because a script test or contract check failed.

Exit Codes

Exit Code	Meaning	When It Happens
0	Success	All executed requests passed, or --no-fail was used.
1	Run failure	At least one executed request, script test, or contract check failed.
2	Fatal error	Invalid command, invalid file, invalid JSON, unsupported report type, missing report output, or unexpected runtime error.

Exit Code Examples

Normal run with failures:

callme run collection.json --env environment.json
# exits 1 if any request/script/contract fails

Collect failures but do not fail the process:

callme run collection.json --env environment.json --no-fail
# exits 0 even if failures exist

Stop early but do not fail the process:

callme run collection.json --env environment.json --bail --no-fail
# stops on first failure, exits 0

Output Modes

Normal Output

Normal mode prints:

• collection header
• request status line
• script test details
• contract check details
• summary
• report path when a report is generated

Example:

Running collection: Registration

[1/8] POST /api/v3/registration/pin .... 200 OK (316ms)
  Script Tests:
    PASS Status code is 200
  Contract Checks:
    PASS Contract: data.last_sms

Summary:
Requests: 8
Passed: 7
Failed: 1

Silent Output

Silent mode prints only:

• request status lines
• summary

It hides:

• collection header
• script test details
• contract check details
• verbose debug logs
• report written message

Command:

callme run collection.json --env environment.json --silent

Example:

[1/8] POST /api/v3/registration/pin .... 200 OK (316ms)
[2/8] POST /api/v2/login .... 200 OK (89ms)

Summary:
Requests: 8
Passed: 7
Failed: 1

Verbose Output

Verbose mode prints normal output plus:

• full resolved request URL
• request headers
• query params
• request body
• response headers
• response body
• environment changes after each request

Command:

callme run collection.json --env environment.json --verbose

Verbose output is useful for developers and QA when comparing CLI behavior with desktop execution.

Important: verbose output may print secrets such as tokens, API keys, cookies, and personal data. Avoid using --verbose in shared CI logs unless logs are protected.

Advanced Flags

--no-fail

Use --no-fail when you want the CLI to complete and produce evidence without failing the shell process.

Behavior:

• failures are still detected
• failures are still printed
• failures are still included in reports
• final exit code is forced to 0 unless a fatal error occurs

Typical use cases:

• nightly diagnostics where reports should be collected
• non-blocking monitoring jobs
• demo runs
• investigation runs in CI where artifacts matter more than gating

Example:

callme run collection.json --env environment.json --no-fail

--bail

Use --bail when later requests depend on earlier successful requests.

Behavior:

• execution stops after the first failed request
• summary includes only executed requests
• reports include only executed requests
• exit code is still 1 if a failure occurred, unless --no-fail is used

Typical use cases:

• authentication flow fails, so later API calls are useless
• registration flow depends on previous tokens
• fast feedback in CI
• reducing noise after the first broken step

Example:

callme run collection.json --env environment.json --bail

--verbose

Use --verbose for debugging.

Behavior:

• disables spinner output
• prints detailed request/response data
• prints environment mutations
• still prints script and contract results

Example:

callme run collection.json --env environment.json --verbose

--silent

Use --silent for compact CI logs.

Behavior:

• prints only request status lines and summary
• hides script details
• hides contract details
• hides verbose logs
• hides report written message
• overrides --verbose

Example:

callme run collection.json --env environment.json --silent

Flag Priority

Combination	Result
--silent --verbose	Silent wins. Verbose logs are hidden.
--bail --no-fail	Execution stops early, but final exit code is 0.
--report ... --silent	Report is still written, but the report path message is hidden.
--report without --output	Fatal error, exit code 2.
--output without --report	Fatal error, exit code 2.

Flag Combination Matrix

The following table describes all combinations of the four execution flags.

Flags	Execution	Output	Exit Code On Run Failure
none	Runs all requests	Normal	1
--no-fail	Runs all requests	Normal	0
--bail	Stops after first failure	Normal	1
--verbose	Runs all requests	Detailed debug	1
--silent	Runs all requests	Minimal	1
--no-fail --bail	Stops after first failure	Normal	0
--no-fail --verbose	Runs all requests	Detailed debug	0
--no-fail --silent	Runs all requests	Minimal	0
--bail --verbose	Stops after first failure	Detailed debug	1
--bail --silent	Stops after first failure	Minimal	1
--verbose --silent	Runs all requests	Minimal because silent wins	1
--no-fail --bail --verbose	Stops after first failure	Detailed debug	0
--no-fail --bail --silent	Stops after first failure	Minimal	0
--no-fail --verbose --silent	Runs all requests	Minimal because silent wins	0
--bail --verbose --silent	Stops after first failure	Minimal because silent wins	1
--no-fail --bail --verbose --silent	Stops after first failure	Minimal because silent wins	0

Reports

Callme CLI supports JSON and HTML reports.

JSON Report

Command:

callme run collection.json --env environment.json --report json --output report.json

Output shape:

{
  "collectionName": "Registration",
  "summary": {
    "total": 8,
    "passed": 7,
    "failed": 1
  },
  "results": []
}

Use JSON reports for:

• CI parsing
• dashboards
• automated trend analysis
• archival evidence

HTML Report

Command:

callme run collection.json --env environment.json --report html --output report.html

The HTML report includes:

• collection name
• summary cards
• expandable request rows
• request URL
• request headers
• query params
• request body
• response headers
• response metadata
• response body
• script test results
• contract check results

Use HTML reports for:

• QA review
• business sharing
• release evidence
• manual investigation

Report And Flag Combinations

Generate a JSON report and fail CI on failures:

callme run collection.json --env environment.json --report json --output report.json

Generate an HTML report and fail CI on failures:

callme run collection.json --env environment.json --report html --output report.html

Generate a report but never fail the process:

callme run collection.json --env environment.json --report html --output report.html --no-fail

Stop early and write a report for executed requests:

callme run collection.json --env environment.json --report html --output report.html --bail

Stop early, write a report, but exit 0:

callme run collection.json --env environment.json --report html --output report.html --bail --no-fail

Write a report with minimal terminal logs:

callme run collection.json --env environment.json --report json --output report.json --silent

Debug and write an HTML report:

callme run collection.json --env environment.json --report html --output report.html --verbose

Common Usage By Role

Business User

Run a flow and generate a readable report:

callme run registration.json --env staging.json --report html --output registration-report.html

Meaning:

• executes the exported business flow
• creates an HTML report
• shows which checks passed or failed
• suitable for sharing with non-developers

QA Engineer

Run regression tests and stop at the first broken step:

callme run regression.json --env qa.json --bail --report html --output regression.html

Meaning:

• stops quickly when the flow becomes invalid
• keeps the failure focused
• writes a report for the executed part of the flow

Developer

Debug variable resolution, request body, response body, and env changes:

callme run collection.json --env local.json --verbose

Meaning:

• shows resolved request URL
• shows headers and body sent by CLI
• shows response payloads
• shows variables written by scripts

DevOps Engineer

Run in CI and fail the pipeline on test or contract failures:

callme run collection.json --env ci.json --silent --report json --output callme-report.json

Meaning:

• compact logs
• machine-readable artifact
• non-zero exit code when the API flow fails

Non-Blocking CI Evidence

Run a collection, collect the report, but do not block the pipeline:

callme run collection.json --env ci.json --silent --no-fail --report html --output callme-report.html

Meaning:

• failures are still recorded
• report is generated
• process exits 0

CI/CD Examples

GitHub Actions

name: API Regression

on:
  pull_request:
  workflow_dispatch:

jobs:
  api-regression:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install CLI
        working-directory: callme_cli
        run: npm ci

      - name: Build CLI
        working-directory: callme_cli
        run: npm run build

      - name: Run API collection
        working-directory: callme_cli
        run: npm run start -- run ./data/register-collection.json --env ./data/environment.json --silent --report json --output report.json

      - name: Upload report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: callme-report
          path: callme_cli/report.json

Non-Blocking Scheduled Check

- name: Run non-blocking API health check
  working-directory: callme_cli
  run: npm run start -- run ./data/register-collection.json --env ./data/environment.json --silent --no-fail --report html --output report.html

Fail Fast In CI

callme run collection.json --env ci.json --silent --bail --report json --output report.json

Meaning:

• stops on first failure
• keeps logs small
• returns 1 on failure
• still writes a report for executed requests

Local Development Commands

Build:

npm run build

Run with sample data:

npm run start -- run ./data/register-collection.json --env ./data/environment.json

Run and generate HTML:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --report html --output report.html

Run and generate JSON:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --report json --output report.json

Run silently:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --silent

Run verbose:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --verbose

Run fail-fast:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --bail

Run without failing the shell:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --no-fail

Troubleshooting

zsh: command not found: callme

The global callme command is not linked or installed.

Use the local development command:

npm run start -- run ./data/register-collection.json --env ./data/environment.json

Or link the CLI globally after building:

npm run build
npm link

Permission Error During npm link

If npm tries to write into /usr/local/lib/node_modules and fails with EACCES, your global npm prefix is not writable by your user.

Recommended options:

• use npm run start -- ... during development
• configure a user-owned npm global prefix
• use a Node version manager that owns its global npm directory

Avoid changing system permissions unless you understand the impact.

Fatal: Collection.name must be a non-empty string

The collection file does not have a supported collection name.

Supported examples:

{
  "name": "Registration",
  "requests": []
}

or:

{
  "collection": {
    "name": "Registration"
  },
  "requests": []
}

Unsupported report type

Only these report types are supported:

• json
• html

Correct:

callme run collection.json --report html --output report.html

Incorrect:

callme run collection.json --report hml --output report.html

--report Requires --output

This is invalid:

callme run collection.json --report html

Use:

callme run collection.json --report html --output report.html

--output Requires --report

This is invalid:

callme run collection.json --output report.html

Use:

callme run collection.json --report html --output report.html

HTTP 200 But Request Is Failed

A request can fail even when HTTP status is 200.

That happens when:

• a script test fails
• a contract check fails

Example:

GET /userinfo .... 200 OK
Contract Checks:
  FAIL pin length expected at least 8, got 7

The CLI treats this as a failed request because the API contract is not satisfied.

Security Notes

Environment files may contain tokens, API keys, passwords, customer data, or other secrets.

Be careful with:

• --verbose, because it prints headers, request body, response body, and environment changes
• HTML reports, because they include request and response details
• JSON reports, because they include raw execution results
• CI artifacts, because they may persist sensitive values

Recommended practices:

• store environment files securely
• avoid committing real secrets
• use secret management in CI
• restrict access to generated reports
• prefer --silent in public logs

Architecture

Callme CLI is split from callman-core.

callman-core owns:

• request execution
• variable resolution
• environment store behavior
• script execution
• contract validation
• per-request core execution logic

Callme CLI owns:

• command parsing
• file loading
• desktop export normalization
• per-request orchestration for flags
• terminal formatting
• reports
• exit codes

This separation keeps the core reusable by CLI, desktop, frontend, and future automation tools.

Recommended Commands

For local debugging:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --verbose

For QA regression:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --bail --report html --output report.html

For CI gating:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --silent --report json --output report.json

For non-blocking CI evidence:

npm run start -- run ./data/register-collection.json --env ./data/environment.json --silent --no-fail --report html --output report.html