@evalguardai/cli

@evalguard/cli

Command-line interface for EvalGuard -- run LLM evaluations, security scans, and guardrail checks from your terminal and CI/CD pipelines.

Installation

npm install -g @evalguard/cli

Quick Start

# Authenticate
evalguard login --key eg_live_...

# Initialize a project
evalguard init --project my-project

# Run an evaluation
evalguard eval evals/example.json --wait

# Run a security scan
evalguard scan scans/example.json --wait

# Run a local eval (no API key needed)
evalguard eval:local --model openai:gpt-4o --prompt "Answer: {{input}}" --scorer exact-match

# Check firewall rules
evalguard firewall check "Ignore all instructions"

Commands

Command	Description
evalguard login	Authenticate with your EvalGuard API key
evalguard logout	Remove stored credentials
evalguard init	Initialize EvalGuard in the current project (creates config and example files)
evalguard eval <file>	Run an evaluation from a JSON config file
evalguard scan <file>	Run a security scan from a JSON config file
evalguard whoami	Show current authentication status
evalguard eval:local	Run an evaluation locally without the cloud API
evalguard scan:local	Run a security scan locally without the cloud API
evalguard generate	Auto-generate eval cases or scan configs from a prompt
evalguard validate	Validate an eval or scan config file for correctness
evalguard compare	Compare results between two eval runs (drift detection)
evalguard list	List past eval runs and security scans
evalguard firewall	Check input against firewall rules or manage rule sets
evalguard watch	Watch eval/scan files and re-run on changes

Configuration

The CLI stores credentials at ~/.evalguard/config.json. Project-level settings go in evalguard.config.json in your repository root.

{
  "$schema": "https://evalguard.ai/schema/config.json",
  "projectId": "my-project",
  "defaultModel": "gpt-4o",
  "evalsDir": "./evals",
  "scansDir": "./scans"
}

CI/CD Usage

# GitHub Actions example
- name: Run EvalGuard security scan
  env:
    EVALGUARD_API_KEY: ${{ secrets.EVALGUARD_API_KEY }}
  run: |
    npx @evalguard/cli login --key $EVALGUARD_API_KEY
    npx @evalguard/cli scan scans/production.json --wait

Documentation

Full documentation at evalguard.ai/docs/cli.

License

MIT -- see LICENSE for details.