computer-pilot-plugin

computer-pilot plugin

Agent plugin for computer-pilot — teach AI agents to control the macOS desktop via CLI.

Install

Step 1: Install cu binary

# Option A: Build from source
git clone https://github.com/relixiaobo/computer-pilot.git
cd computer-pilot && cargo build --release
sudo cp target/release/cu /usr/local/bin/

# Option B: Download prebuilt (macOS Apple Silicon)
curl -L https://github.com/relixiaobo/computer-pilot/releases/latest/download/cu-arm64 -o /usr/local/bin/cu
chmod +x /usr/local/bin/cu

Step 2: Grant permissions

cu setup

Step 3: Install plugin in Claude Code

/install-plugin computer-pilot-plugin

What it does

Adds a skill that teaches Claude Code how to use cu to:

• Observe: snapshot UI elements (AX tree), OCR, screenshots
• Act: click, type, keyboard shortcuts, scroll, drag
• Automate: open apps, navigate menus, handle dialogs, file operations

Usage

Once installed, Claude Code automatically uses cu when you ask it to interact with desktop apps:

"Open Calculator and compute 2+3"
"Take a screenshot of Chrome"
"Open System Settings and enable Dark Mode"

Or use the /desktop command:

/desktop open Finder and create a new folder called "test"

Commands (12)

Command	What it does
cu setup	Check permissions
cu apps	List running apps
cu snapshot	AX tree with [ref] numbers
cu click	Click by ref or coordinates
cu key	Keyboard shortcut
cu type	Type text
cu scroll	Scroll
cu hover	Move mouse
cu drag	Drag
cu screenshot	Window capture
cu ocr	Vision OCR
cu wait	Wait for condition

Links

• GitHub
• cu --help