Codemod CLI
The Codemod CLI is a command-line interface for running and managing Butterflow workflows - a lightweight, self-hostable workflow engine designed for running large-scale code transformation jobs.
NOTE: This CLI is currently in alpha and may change over time. So be careful when using it in production environments.
For more information read the blog post announcing the CLI.
And any feedback is welcome!
Installation
Building from Source
git clone https://github.com/codemod-com/codemod.git
cd codemod
cargo build --release
From npm registry
npm install -g codemod
Quick Start
- Create a workflow file
workflow.yaml
:
version: "1"
nodes:
- id: hello-world
name: Hello World
type: automatic
runtime:
type: direct
steps:
- id: hello
name: Say Hello
run: echo "Hello, World!"
codemod run -w workflow.yaml
Commands
codemod init
Initialize a new Butterflow workflow project with interactive setup:
codemod init
codemod init my-project/
codemod init --name my-workflow
Options:
--name <NAME>
: Set the project name (defaults to directory name)
--force
: Overwrite existing files if they exist
- Positional argument: Target directory path (defaults to current directory)
Interactive Setup:
The init command will prompt you with several questions to customize your project:
-
Project Type Selection:
? What type of workflow would you like to create?
❯ AST-grep with JavaScript/TypeScript rules
AST-grep with YAML rules
Blank workflow (custom commands)
-
Language Selection (if AST-grep is chosen):
? Which language would you like to target?
❯ JavaScript/TypeScript
Python
Rust
Go
Java
C/C++
Other
-
Project Configuration:
? Project name: my-codemod
? Description: Converts legacy API calls to new format
? Author: Your Name <your.email@example.com>
Generated Project Structure:
Depending on your selections, the init command creates different project templates:
AST-grep JavaScript Project
my-codemod/
├── workflow.yaml # Main workflow definition
├── package.json # Node.js dependencies
├── rules/
│ └── example-rule.js # JavaScript AST transformation rules
├── scripts/
│ └── apply-codemod.js # Script to apply transformations
├── tests/
│ ├── input/ # Test input files
│ └── expected/ # Expected output files
└── README.md # Project documentation
AST-grep YAML Project
my-codemod/
├── workflow.yaml # Main workflow definition
├── rules/
│ └── example-rule.yml # YAML AST pattern rules
├── scripts/
│ └── apply-rules.sh # Shell script to apply rules
├── tests/
│ ├── input/ # Test input files
│ └── expected/ # Expected output files
└── README.md # Project documentation
Blank Workflow Project
my-workflow/
├── workflow.yaml # Basic workflow template
├── scripts/
│ └── example.sh # Example script
└── README.md # Project documentation
Example Usage:
$ codemod init my-js-codemod
? What type of workflow would you like to create? AST-grep with JavaScript/TypeScript rules
? Which language would you like to target? JavaScript/TypeScript
? Project name: my-js-codemod
? Description: Migrate from React class components to hooks
? Author: John Doe <john@example.com>
✓ Created workflow.yaml
✓ Created package.json with @ast-grep/cli dependency
✓ Created rules/migrate-to-hooks.js
✓ Created scripts/apply-codemod.js
✓ Created test structure
✓ Created README.md
Next steps:
cd my-js-codemod
npm install
codemod validate -w workflow.yaml
codemod run -w workflow.yaml
codemod run
Execute a workflow from various sources:
codemod run -w workflow.yaml
codemod run -w path/to/workflow.yaml
codemod run ./my-workflow-bundle/
Options:
-w, --workflow <FILE>
: Path to the workflow definition file
- Positional argument: Path to workflow file or bundle directory
codemod resume
Resume a paused workflow or trigger manual tasks:
codemod resume -i <workflow-run-id>
codemod resume -i <workflow-run-id> -t <task-uuid>
codemod resume -i <workflow-run-id> --trigger-all
Options:
-i, --id <UUID>
: Workflow run ID to resume
-t, --task <UUID>
: Specific task UUID to trigger
--trigger-all
: Trigger all tasks in AwaitingTrigger
state
codemod validate
Validate a workflow definition without executing it:
codemod validate -w workflow.yaml
codemod validate -w path/to/workflow.yaml
codemod validate ./my-workflow-bundle/
Validation Checks:
- Schema validation
- Node ID uniqueness
- Step ID uniqueness within nodes
- Template ID uniqueness
- Dependency validation
- Cyclic dependency detection
- Template reference validation
- Matrix strategy validation
- State schema validation
- Variable reference syntax validation
Example Output:
Valid workflow:
$ codemod validate -w valid-workflow.yaml
✓ Workflow definition is valid
Schema validation: Passed
Node dependencies: Valid (3 nodes, 2 dependency relationships)
Template references: Valid (2 templates, 3 references)
Matrix strategies: Valid (1 matrix node)
State schema: Valid (2 schema definitions)
Invalid workflow:
$ codemod validate -w invalid-workflow.yaml
✗ Workflow definition is invalid
Error at nodes[2].strategy: Matrix strategy requires 'values' or 'from_state' property
Error at nodes[1].depends_on[0]: Referenced node 'non-existent-node' does not exist
Error: Cyclic dependency detected: node-a → node-b → node-a
codemod login
Authenticate with a Butterflow registry to publish and manage codemods:
codemod login
codemod login --registry https://registry.example.com
codemod login --token <your-token>
codemod login --username <username>
Options:
--registry <URL>
: Registry URL (defaults to official Butterflow registry)
--token <TOKEN>
: Authentication token (for non-interactive login)
--username <USERNAME>
: Username for interactive login
--scope <SCOPE>
: Organization or user scope for publishing
Interactive Login:
$ codemod login
? Registry URL: https://app.butterflow.com(default)
? Username: john.doe
? Password: ********
? Organization (optional): my-org
✓ Successfully logged in as john.doe
✓ Default publish scope set to @my-org
Token-based Login (for CI/CD):
export BUTTERFLOW_TOKEN="your-registry-token"
codemod login --token $BUTTERFLOW_TOKEN
codemod publish
Publish a codemod to a registry for sharing and reuse:
codemod publish
codemod publish ./my-codemod/
codemod publish --version 1.2.3
codemod publish --registry https://registry.example.com
codemod publish --dry-run
Options:
--version <VERSION>
: Explicit version (overrides manifest version)
--registry <URL>
: Target registry URL
--tag <TAG>
: Tag for the release (e.g., beta
, latest
)
--access <LEVEL>
: Access level (public
or private
)
--dry-run
: Validate and pack without uploading
--force
: Override existing version (use with caution)
Publishing Flow:
$ codemod publish
✓ Validating codemod.yaml manifest
✓ Validating workflow.yaml
✓ Running tests (if present)
✓ Building codemod bundle
✓ Uploading to registry @my-org/react-hooks-migration@1.0.0
✓ Published successfully!
Install with: codemod run @my-org/react-hooks-migration@1.0.0
Codemod Manifest Standard
Published codemods must include a codemod.yaml
manifest file that defines metadata, dependencies, and publishing information.
Manifest Schema
name: react-hooks-migration
version: 1.0.0
description: Migrates React class components to functional components with hooks
author: John Doe <john@example.com>
license: MIT
repository: https://github.com/user/react-hooks-migration
registry:
access: public
targets:
languages:
- javascript
- typescript
frameworks:
- react
versions:
react: ">=16.8.0"
Required Fields
name | string | Codemod package name (must be unique in scope) |
version | string | Semantic version (e.g., "1.0.0") |
description | string | Brief description of what the codemod does |
author | string | Author name and email |
Optional Fields
license | string | License identifier (e.g., "MIT", "Apache-2.0") |
repository | string | Repository URL for the codemod source code |
registry.access | string | Access level: "public" or "private" |
targets.languages | array | Supported programming languages |
targets.frameworks | array | Supported frameworks or libraries |
targets.versions | object | Version constraints for frameworks |
Publishing Examples
Basic Codemod
name: remove-console-logs
version: 1.0.0
description: Removes console.log statements from JavaScript/TypeScript files
author: Developer <dev@example.com>
license: MIT
repository: https://github.com/dev/remove-console-logs
targets:
languages: [javascript, typescript]
Framework-Specific Codemod
name: api-migration-v2
version: 2.1.0
description: Migrates legacy API calls to new v2 endpoints
author: API Team <api-team@company.com>
license: Apache-2.0
repository: https://github.com/company/api-migration-v2
registry:
access: private
targets:
languages: [javascript, typescript]
frameworks: [react, vue, angular]
versions:
react: ">=16.0.0"
vue: ">=3.0.0"
Workflow Sources
The CLI supports loading workflows from different sources:
File Path
codemod run -w path/to/your/workflow.yaml
Loads the specific file. The base path for execution is the directory containing this file.
Directory Path (Bundle)
codemod run path/to/your/bundle/
Butterflow looks for a standard workflow file (e.g., workflow.yaml
) inside this directory. The base path for execution is this directory.
Registry Identifier
codemod run @my-org/react-hooks-migration@1.0.0
codemod run @my-org/react-hooks-migration@latest
codemod run @my-org/api-migration-v2@2.1.0 --param api_base_url=https://staging-api.example.com
When running from a registry, Butterflow:
- Downloads the codemod bundle from the registry
- Caches it locally for faster subsequent runs
- Validates the manifest and workflow
- Executes with the bundle directory as the base path
For your information the defautl registry is https://app.codemod.com/
. and you can visualize codemods on the Codemod Registry webapp.
Workflow Bundles
A workflow bundle is a directory containing:
- The main workflow definition file (e.g.,
workflow.yaml
)
- Any scripts, binaries, or configuration files referenced by the workflow steps
- Additional assets needed by the tasks
When running from a directory, Butterflow uses that directory as the root for resolving relative paths during execution.
Runtime Execution
Container Runtimes
The CLI supports multiple execution runtimes:
- Docker: Uses Docker daemon for container execution
- Podman: Uses Podman for container execution
- Direct: Runs commands directly on the host machine
When using runtime: direct
, commands execute with the same working directory as the codemod
CLI invocation. Use the $CODEMOD_PATH
environment variable to access files within the workflow bundle.
Environment Variables
The CLI provides several environment variables to running tasks:
$STATE_OUTPUTS
: File descriptor for writing state updates
$CODEMOD_PATH
: Absolute path to the workflow bundle root (for direct
runtime)
$BUTTERFLOW_STATE
: Path to state file for programmatic access
Manual Triggers and Resumption
Manual Nodes
Nodes with type: manual
or trigger: { type: manual }
will pause execution and await manual triggers:
nodes:
- id: deploy-prod
name: Deploy to Production
type: automatic
trigger:
type: manual
steps:
- id: deploy
run: ./deploy.sh production
Triggering Manual Tasks
When a workflow pauses for manual triggers:
- The workflow state is persisted with tasks marked
AwaitingTrigger
- The CLI provides task UUIDs for manual triggering
- Use
codemod resume
to trigger specific tasks or all awaiting tasks
codemod resume -i abc123-workflow-run-id -t def456-task-uuid
codemod resume -i abc123-workflow-run-id --trigger-all
Error Handling
Automatic Validation
Validation is performed automatically when running workflows:
$ codemod run -w invalid-workflow.yaml
✗ Workflow definition is invalid
Error: Cyclic dependency detected: node-a → node-b → node-a
Workflow execution aborted
Resume After Failures
If a workflow fails or is interrupted:
- The state is automatically persisted
- Use
codemod resume
to continue from the last checkpoint
- Failed tasks can be retried while preserving completed work
Examples
Basic Workflow Execution
cat > hello.yaml << EOF
version: "1"
nodes:
- id: greet
name: Greeting
type: automatic
runtime:
type: direct
steps:
- id: hello
run: echo "Hello from Butterflow!"
EOF
codemod run -w hello.yaml
Matrix Workflow with Manual Approval
codemod run -w deploy-workflow.yaml
codemod resume -i <run-id> -t <staging-task-uuid>
codemod resume -i <run-id> -t <prod-task-uuid>
Workflow Validation and Publishing
codemod validate -w complex-workflow.yaml
codemod run -w complex-workflow.yaml
codemod login
codemod publish ./my-codemod/
codemod run @my-org/my-codemod@1.0.0
Publishing Workflow
codemod init my-api-migration
cd my-api-migration
cat > codemod.yaml << EOF
name: my-api-migration
version: 1.0.0
description: Migrates legacy API calls
author: Developer <dev@example.com>
workflow: workflow.yaml
targets:
languages: [javascript, typescript]
EOF
codemod validate
codemod publish --dry-run
codemod publish
Global Options
Most commands support these global options:
--help, -h
: Show help information
--version, -V
: Show version information
--verbose, -v
: Enable verbose output
--quiet, -q
: Suppress non-essential output
For detailed help on any command, use:
codemod <command> --help
JSSG (JavaScript AST-grep)
JSSG is a JavaScript/TypeScript codemod runner and testing framework inspired by ast-grep. It enables you to write codemods in JavaScript and apply them to codebases with powerful CLI and test automation support.
Running a Codemod
To run a JavaScript codemod on a target directory:
codemod jssg run my-codemod.js ./src --language javascript
Options:
--language <LANG>
: Target language (javascript, typescript, etc.)
--extensions <ext1,ext2>
: Comma-separated list of file extensions to process
--no-gitignore
: Do not respect .gitignore files
--include-hidden
: Include hidden files and directories
--max-threads <N>
: Maximum number of concurrent threads
--dry-run
: Perform a dry run without making changes
See codemod jssg run --help
for all options.
Example
codemod jssg run my-codemod.js ./src --language javascript --dry-run
JSSG Testing Framework Usage Guide
Overview
The JSSG Testing Framework provides comprehensive testing capabilities for JavaScript codemods with before/after fixture files. It integrates with the existing ExecutionEngine and provides a familiar test runner interface.
Quick Start
1. Basic Test Structure
Create a test directory with the following structure:
tests/
├── simple-transform/
│ ├── input.js
│ └── expected.js
├── complex-case/
│ ├── input.ts
│ └── expected.ts
└── multi-file/
├── input/
│ ├── file1.js
│ └── file2.js
└── expected/
├── file1.js
└── file2.js
2. Running Tests
codemod jssg test my-codemod.js --language javascript
codemod jssg test my-codemod.js --language typescript --test-directory ./my-tests
codemod jssg test my-codemod.js --language javascript --update-snapshots
codemod jssg test my-codemod.js --language javascript --verbose
codemod jssg test my-codemod.js --language javascript --watch
CLI Options
Required Arguments
codemod_file
: Path to the codemod JavaScript file
--language
: Target language (javascript, typescript, etc.)
Test Discovery
--test-directory
: Test directory (default: "tests")
--filter
: Run only tests matching pattern
Output Control
--reporter
: Output format (console, json, terse)
--verbose
: Show detailed output
--context-lines
: Number of diff context lines (default: 3)
--ignore-whitespace
: Ignore whitespace in comparisons
Test Execution
--timeout
: Test timeout in seconds (default: 30)
--max-threads
: Maximum concurrent threads
--sequential
: Run tests sequentially
--fail-fast
: Stop on first failure
Snapshot Management
--update-snapshots
: Create/update expected files
--expect-errors
: Comma-separated patterns for tests expected to fail
Development
--watch
: Watch for changes and re-run tests
Test File Formats
Single File Format
Each test case is a directory with input.{ext}
and expected.{ext}
files:
test-case-name/
├── input.js # Input code
└── expected.js # Expected output
Multi-File Format
For testing multiple files, use input/
and expected/
directories:
test-case-name/
├── input/
│ ├── file1.js
│ └── file2.js
└── expected/
├── file1.js
└── file2.js
Language Support
The framework automatically detects input files based on language extensions:
- JavaScript:
.js
, .jsx
, .mjs
- TypeScript:
.ts
, .tsx
, .mts
- Other languages: Determined by
get_extensions_for_language()
Error Handling
Missing Expected Files
codemod jssg test my-codemod.js --language javascript
codemod jssg test my-codemod.js --language javascript --update-snapshots
Ambiguous Input Files
Expected Test Failures
codemod jssg test my-codemod.js --language javascript --expect-errors "error-case,invalid-syntax"
Output Formats
Console (Default)
test my-test ... ok
test failing-test ... FAILED
failures:
---- failing-test stdout ----
Output mismatch for file expected.js:
-const x = 1;
+const y = 1;
test result: FAILED. 1 passed; 1 failed; 0 ignored
JSON
codemod jssg test my-codemod.js --language javascript --reporter json
{
"type": "suite",
"event": "started",
"test_count": 2
}
{
"type": "test",
"event": "started",
"name": "my-test"
}
{
"type": "test",
"name": "my-test",
"event": "ok"
}
Terse
codemod jssg test my-codemod.js --language javascript --reporter terse
..F
failures:
...
Advanced Features
Snapshot Management
codemod jssg test my-codemod.js --language javascript --update-snapshots
codemod jssg test my-codemod.js --language javascript --filter "specific-test" --update-snapshots
Test Filtering
codemod jssg test my-codemod.js --language javascript --filter "transform"
codemod jssg test my-codemod.js --language javascript --filter "my-specific-test"
Performance Tuning
codemod jssg test my-codemod.js --language javascript --max-threads 2
codemod jssg test my-codemod.js --language javascript --sequential
codemod jssg test my-codemod.js --language javascript --timeout 60
Diff Customization
codemod jssg test my-codemod.js --language javascript --context-lines 5
codemod jssg test my-codemod.js --language javascript --ignore-whitespace
Integration with Development Workflow
CI/CD Integration
- name: Run codemod tests
run: codemod jssg test my-codemod.js --language javascript --reporter json
IDE Integration
The framework outputs standard test results that can be consumed by IDEs and test runners.
Watch Mode Development
codemod jssg test my-codemod.js --language javascript --watch --verbose
Best Practices
- Organize tests by functionality: Group related test cases in descriptive directories
- Use meaningful test names: Directory names become test names in output
- Start with --update-snapshots: Generate initial expected files, then review and commit
- Use --expect-errors for negative tests: Test error conditions explicitly
- Leverage --filter during development: Focus on specific tests while developing
- Use --watch for rapid iteration: Get immediate feedback on changes
Troubleshooting
Common Issues
- No tests found: Check test directory path and file extensions
- Ambiguous input files: Ensure only one input file per test case
- Timeout errors: Increase timeout for complex codemods
- Memory issues: Reduce max-threads for large test suites
Debug Mode
codemod jssg test my-codemod.js --language javascript --verbose --sequential
This framework provides a robust foundation for testing JavaScript codemods with familiar tooling and comprehensive features.