agent-device

agent-device

agent-device is a CLI for UI automation on iOS, tvOS, macOS, Android, and AndroidTV. It is designed for agent-driven workflows: inspect the UI, act on it deterministically, and keep that work session-aware and replayable.

If you know Vercel's agent-browser, this project applies the same broad idea to mobile apps and devices.

Project Goals

• Give agents a practical way to understand mobile UI state through structured snapshots.
• Keep automation flows token-efficient enough for real agent loops.
• Make common interactions reliable enough for repeated automation runs.
• Keep automation grounded in sessions, selectors, and replayable flows instead of one-off scripts.

Core Ideas

• Sessions: open a target once, interact within that session, then close it cleanly.
• Snapshots: inspect the current accessibility tree in a compact form and get stable refs for exploration.
• Refs vs selectors: use refs for discovery, use selectors for durable replay and assertions.
• Human docs vs agent skills: docs explain the system for people; skills provide compact operating guidance for agents.

Command Flow

The canonical loop is:

agent-device open SampleApp --platform ios
agent-device snapshot -i
agent-device press @e3
agent-device diff snapshot -i
agent-device fill @e5 "test"
agent-device close

In practice, most work follows the same pattern:

• open a target app or URL.
• snapshot -i to inspect the current screen.
• press, fill, scroll, get, or wait using refs or selectors.
• diff snapshot or re-snapshot after UI changes.
• close when the session is finished.

Where To Go Next

For people:

• Website
• Docs

For agents:

• agent-device skill
• dogfood skill
• agent-device skill on ClawHub

Install

npm install -g agent-device

On macOS, agent-device includes a local agent-device-macos-helper source package that is built on demand for desktop permission checks, alert handling, and helper-backed desktop snapshot surfaces. Release distribution should use a signed/notarized helper build; source checkouts fall back to a local Swift build.

Contributing

See CONTRIBUTING.md.

Made at Callstack

agent-device is an open source project and will always remain free to use. Callstack is a group of React and React Native geeks. Contact us at [e-mail] if you need any help with these technologies or just want to say hi.