@evalguardai/cli

@evalguard/cli

Command-line interface for EvalGuard -- run LLM evaluations, security scans, and guardrail checks from your terminal and CI/CD pipelines.

Installation

npm install -g @evalguard/cli

Quick Start

# Authenticate
evalguard login --key eg_live_...

# Initialize a project
evalguard init --project my-project

# Run an evaluation
evalguard eval evals/example.json --wait

# Run a security scan
evalguard scan scans/example.json --wait

# Run a local eval (no API key needed)
evalguard eval:local --model openai:gpt-4o --prompt "Answer: {{input}}" --scorer exact-match

# Check firewall rules
evalguard firewall check "Ignore all instructions"

Commands

Command	Description
evalguard login	Authenticate with your EvalGuard API key
evalguard logout	Remove stored credentials
evalguard init	Initialize EvalGuard in the current project (creates config and example files)
evalguard eval <file>	Run an evaluation from a JSON config file
evalguard scan <file>	Run a security scan from a JSON config file
evalguard whoami	Show current authentication status
evalguard eval:local	Run an evaluation locally without the cloud API
evalguard scan:local	Run a security scan locally without the cloud API
evalguard generate	Auto-generate eval cases or scan configs from a prompt
evalguard validate	Validate an eval or scan config file for correctness
evalguard compare	Compare results between two eval runs (drift detection)
evalguard list	List past eval runs and security scans
evalguard firewall	Check input against firewall rules or manage rule sets
evalguard watch	Watch eval/scan files and re-run on changes

Configuration

The CLI stores credentials at ~/.evalguard/config.json. Project-level settings go in evalguard.config.json in your repository root.

{
  "$schema": "https://evalguard.ai/schema/config.json",
  "projectId": "my-project",
  "defaultModel": "gpt-4o",
  "evalsDir": "./evals",
  "scansDir": "./scans"
}

CI/CD Usage

# GitHub Actions example
- name: Run EvalGuard security scan
  env:
    EVALGUARD_API_KEY: ${{ secrets.EVALGUARD_API_KEY }}
  run: |
    npx @evalguard/cli login --key $EVALGUARD_API_KEY
    npx @evalguard/cli scan scans/production.json --wait

Documentation

Full documentation at evalguard.ai/docs/cli.

License

Apache License, Version 2.0 — see LICENSE and NOTICE.

This CLI is a thin public client for the EvalGuard service. It is not covered by any proprietary license restriction — you may fork, modify, and redistribute it under Apache 2.0. The EvalGuard service itself, its backend engine, scorers, and attack plugins are proprietary software operated as a hosted service and are NOT covered by Apache 2.0. Access is governed by the EvalGuard Terms of Service.

"EvalGuard" is a trademark of EvalGuard, Inc. Derivative forks must not use the EvalGuard name or logo to imply endorsement or drop-in compatibility with the hosted service.