@testivai/witness-cli

@testivai/witness-cli

Universal visual regression testing CLI - Record visual tests without writing code, run them anywhere.

Overview

The TestivAI CLI enables visual regression testing for any test framework (Selenium, Cypress, etc.) using the Sidecar Pattern. Instead of building separate SDKs for each framework, this CLI runs alongside your existing tests and uses the testivai.witness() function to capture visual evidence.

Installation

# Install CLI globally
npm install -g @testivai/witness-cli

# Initialize your project (installs all dependencies)
testivai init

Dependencies

The TestivAI CLI requires:

• @playwright/test ^1.40.0 - Test runner framework
• playwright ^1.40.0 - Browser automation
• @testivai/witness-playwright - TestivAI reporter (for batch uploads)

Note: These dependencies are automatically installed when you run testivai init. In CI/CD environments, browsers are not auto-installed - you'll need to run npx playwright install chromium.

Quick Start

# 1. Initialize your project
testivai init

# 2. Authenticate
testivai auth <your-api-key>

# 3. Record a visual test (no code required!)
testivai record https://staging.myapp.com

# 4. Run tests
testivai run

Commands

testivai init

Initialize TestivAI in your project.

testivai init

Options:

• -f, --force - Overwrite existing configuration

This command will:

• Install Playwright and TestivAI reporter dependencies
• Create testivai.config.ts configuration file
• Create visual-tests/ directory
• Install Playwright browsers

testivai auth <api-key>

Authenticate with your TestivAI API key.

testivai auth tstvai-abc123xyz

Or use environment variable:

export TESTIVAI_API_KEY=tstvai-abc123xyz
testivai auth

testivai record <url>

Record a visual test by interacting with your app.

# Single page recording
testivai record https://staging.myapp.com/checkout

# Multi-page recording (mark pages during recording)
testivai record https://staging.myapp.com --multi-page

Options:

• -o, --output <file> - Custom output filename
• --no-inject - Output raw Playwright code without TestivAI injection
• --multi-page - Enable multi-page capture mode

Multi-Page Recording

When using --multi-page mode:

• A browser opens with your app
• Navigate through your application
• Press Ctrl+Shift+W to mark a page for capture
• Enter a descriptive name (e.g., "login-page", "checkout-form")
• Continue navigating and marking pages
• Close the browser when finished

The CLI generates a test file with all marked pages:

test.describe('Multi-page visual test', () => {
  test('captures marked pages', async ({ page }) => {
    // Page 1: login-page
    await page.goto('https://staging.myapp.com/login');
    await testivai.witness(page, testInfo, 'login-page');
    await page.waitForTimeout(1000);
    
    // Page 2: dashboard
    await page.goto('https://staging.myapp.com/dashboard');
    await testivai.witness(page, testInfo, 'dashboard');
    await page.waitForTimeout(1000);
    
    // Page 3: checkout-form
    await page.goto('https://staging.myapp.com/checkout');
    await testivai.witness(page, testInfo, 'checkout-form');
    await page.waitForTimeout(1000);
  });
});

testivai run [files]

Execute visual tests and upload results to TestivAI.

# Run all tests
testivai run

# Run specific test
testivai run visual-tests/checkout.spec.ts

# Run with verbose output
testivai run --verbose

# Run in CI/CD (quiet mode)
testivai run --quiet

Options:

• --verbose - Show detailed Playwright output
• --quiet - Suppress output (ideal for CI/CD)
• --update-baselines - Auto-approve all visual diffs as new baselines

Batch Uploads

The CLI automatically uploads all screenshots in a batch to TestivAI:

• Multiple screenshots from a single test run are grouped together
• Receives a single Batch ID for the entire test run
• Results appear together in your TestivAI dashboard

Example output:

🧪 Running 3 visual test(s)...

✅ All visual tests passed (3 passed, 12.3s)
📊 Batch ID: 123e4567-e89b-12d3-a456-426614174000
   View results at: https://dashboard.testiv.ai

Global Options

Flag	Description
-v, --version	Print CLI version
-h, --help	Show help
--verbose	Detailed output
-q, --quiet	Suppress banner (for CI)
--debug	Debug mode

Configuration

testivai.config.ts

export default {
  // Base URL (optional)
  baseUrl: 'https://staging.myapp.com',
  
  // Output directory for tests
  outputDir: './visual-tests',
  
  // Viewport settings
  viewport: {
    width: 1280,
    height: 720,
  },
};

Environment Variables

Variable	Purpose
TESTIVAI_API_KEY	API key for authentication
SKIP_BROWSER_INSTALL	Set to 1 to skip automatic browser installation

CI/CD Integration

GitHub Actions

- name: Run Visual Tests
  env:
    TESTIVAI_API_KEY: ${{ secrets.TESTIVAI_API_KEY }}
  run: testivai run --quiet

Jenkins

stage('Visual Tests') {
  environment {
    TESTIVAI_API_KEY = credentials('testivai-api-key')
    SKIP_BROWSER_INSTALL = '1'
  }
  steps {
    sh 'npm install @testivai/witness-cli'
    sh 'npx playwright install chromium'
    sh 'testivai run --quiet'
  }
}

Exit Codes

Code	Meaning
0	Success - all tests passed
1	Test failed - visual diff detected
2	Configuration error
3	Network error
4	Runtime error

Troubleshooting

"Playwright not found" Error

If you see this error when running testivai run:

❌ Playwright not found!

Solution:

# Option 1: Use the init command (recommended)
testivai init

# Option 2: Manual installation
npm install --save-dev @playwright/test playwright
npx playwright install

"Command not found: testivai"

Solution:

# Install globally
npm install -g @testivai/witness-cli

# Or use npx
npx @testivai/witness-cli run

Browsers not installed in CI

Solution: Add this to your CI script:

npx playwright install chromium --with-deps

API Key Issues

Solution:

# Check if API key is set
echo $TESTIVAI_API_KEY

# Or re-authenticate
testivai auth <your-api-key>

For more help, visit: https://docs.testiv.ai

The Sidecar Pattern

Instead of rewriting your Selenium/Cypress tests, use TestivAI as a "sidecar":

• Your Test runs business logic (login, navigate, etc.)
• TestivAI captures visual state at the end
• Results appear in the TestivAI dashboard

This allows you to add visual regression testing to any framework without changing your existing tests.

License

MIT