sim-mirror

SimMirror

Mirror and drive the iOS Simulator from Claude Code, CLI agents and the browser.

A live iOS Simulator in any browser tab or web page, an animated cursor that shows exactly what your AI agent is about to tap, and token-efficient UI snapshots so agents read the screen as compact text instead of screenshots. Works with Claude Code, Codex, Cursor and any other MCP client.

Status: 1.2. The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every 1.x release to what SimMirror promises not to break. 1.2 drives simulators with SimMirror's own helper, reads a screen from its pixels when accessibility says nothing, and lets an app share its own views.

Why SimMirror

• See the simulator anywhere. A live H.264 or JPEG stream in a Chrome tab, an iframe or a <sim-mirror> web component, with touch, scrolling, typing, hardware buttons and light and dark mode.
• Watch what the agent does. Every agent gesture is announced before it lands, and the viewer's cursor glides to the spot first -- so you can follow along, and step in: an agent waits for your hand to be still.
• Spend fewer tokens. Agents read accessibility snapshots with stable element refs and diffs between screens, act in batches with waits, and reach for a screenshot only when a question is visual.
• Read screens accessibility cannot. A game or a canvas reads from its pixels, and an app under development can share its own views, so icon-only buttons and hand-drawn controls read by name.

Install

Claude Code, with the plugin (tools, a skill that teaches snapshot-first use, and /sim-mirror:open):

/plugin marketplace add AndrewKochulab/sim-mirror
/plugin install sim-mirror@sim-mirror

Any MCP client, as a stdio server:

claude mcp add sim-mirror -- uvx --from sim-mirror==1.2.0 sim-mirror mcp

Configurations for Codex, Cursor and other clients.

The command, for sim-mirror open, doctor and the rest:

uv tool install sim-mirror==1.2.0
# or
brew install andrewkochulab/tap/sim-mirror

The viewer library, for your own pages: npm install @andrewkochulab/sim-mirror (npm). SimMirror itself is on PyPI and in a Homebrew tap.

Requirements: an Apple Silicon Mac, Xcode 26 or later, and uv or Homebrew. Nothing else: SimMirror drives a simulator with its own helper, which comes in the package. Details in Installation.

Quickstart

sim-mirror doctor     # check this Mac, ending with a real tap on a simulator
sim-mirror open       # this project's simulator, live in a browser tab

Then ask your agent:

Open Settings → General → About and tell me which iOS version this simulator runs. Use sim_snapshot rather than screenshots.

Getting started explains what happens along the way.

Features

Live viewer. H.264 through WebCodecs where the page can decode it, JPEG otherwise, negotiated per connection. Touch, drag, scroll, typing and paste, Home and Lock, light and dark, and a device picker. It reconnects by itself after a blip and says plainly when the simulator is off.

Agent cursor. Every agent gesture -- tap, long press, swipe, drag, typing -- reaches the viewer before it lands, and the viewer's own pointer glides there first (agent.cursor_lead_ms). Never the Mac's pointer.

Screen understanding. A snapshot is a few lines -- e2 button "General" (201,319) -- with refs that stay with their elements across scrolls, a digest, and diffs. sim_act plays up to 20 steps in one call and waits for text, its absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. See Screen understanding.

Reading pixels. A screen whose accessibility says nothing -- a game, a canvas, an app still loading -- reads as the text macOS's Vision finds in it, with refs to tap and text to wait for, and the viewer can outline what was read (perception.ocr). See Read from pixels.

App SDK. An app under development can link SimMirrorKit, a small Swift package, and call SimMirror.start(): in a Debug build on the simulator it shares its own UIKit and SwiftUI views, so an icon-only button, a card with a tap gesture or a hand-drawn control reads by name, and .simMirror("Settings") names a SwiftUI view. Optional, and nothing of it is in a Release build. See The app SDK.

Connectors. SimMirror's own native helper gives full control with nothing to install; idb_companion still can, and the simctl connector mirrors the screen view-only when neither is there, and says why. auto uses the fastest that works and falls back when one fails. More connectors plug in through an entry point. See Connectors.

Embedding. An iframe with a one-shot ticket, the <sim-mirror> element themed with CSS custom properties, or SimMirror inside a Python application with its own sign-in.

Settings panel. sim-mirror open --settings puts every setting a click away in the viewer, a tab per section, saved for one project or all of them and applied before it says so. Settings that decide what runs or who may reach the daemon wait for a code from sim-mirror settings confirm. See The settings panel.

Doctor. Checks Xcode, its Simulator frameworks, runtimes, the native helper and idb_companion, Device Hub and the desktop session, the text reader and an app's shared hierarchy, then proves a tap reaches a simulator. See the doctor.

Security. Loopback only, a Host allowlist, exact Origin checks, CORS and framing only for origins you list, hashed scoped tokens, and one-shot codes in URL fragments. See Security.

Build and test. sim_build_run and sim_test build with xcodebuild for the agent's simulator, install and launch, and answer with only what failed and where -- each failing test named so it can be run again alone, and on another simulator when asked. Off by default, since they run commands: sim-mirror config set build.tools true.

Measure the token cost

benchmarks/tool_budget.py calls each read-only tool through the same MCP relay a client uses and prints each answer's bytes, estimated tokens and latency on your own screens; see the token budget example. On an iPhone 17 Pro simulator (iOS 26.5, idb connector) showing Settings → General, 5 calls each (2026-09-16):

Call	Tool	Bytes	Tokens (est.)	p50 ms	p95 ms
device info	sim_device	265	46	2.9	3.6
snapshot, full	sim_snapshot	561	112	64.8	76.4
snapshot, diff of an unchanged screen	sim_snapshot	83	5	63.2	64.8
screenshot, 400 px wide	sim_screenshot	62222	469	8.2	8.9
screenshot, 1200 px wide	sim_screenshot	276472	4181	20.4	21.9

Bytes are the whole answer, an image's base64 included; tokens are estimates (text at about 4 characters a token, an image at about 750 pixels a token).

Compatibility

	Expected to work
macOS	26 (15 with Xcode 16.4, view-only)
Xcode	26 and 27
Connectors	native helper (full control), idb_companion 1.5 (full control), simctl (view-only)
App SDK	Debug builds on the iOS Simulator, iOS 16 or later
Clients	Claude Code, Codex, Cursor, any stdio MCP client
Browsers	Chrome, Safari, Firefox, Edge

Rows checked on real hardware are marked verified, with the date, in Compatibility.

From 1.0 the tools, the protocol, the embedding API, the viewer package, settings and the command line are held to what SimMirror promises not to break: a 1.x release adds, and never breaks. Connectors are the exception, and pin a minor version.

Documentation

Everything: getting started, clients, embedding, screen understanding, connectors, security, architecture, and the reference for tools, configuration, the command line and the protocol.

Roadmap

Milestone	What it brings
v0.1 (preview)	Live viewer, agent cursor, person and agent control, MCP tools, sim-mirror doctor, idb and simctl connectors, build and test as a preview
v0.2	Build, run and test tools, a settings panel, Xcode 27's UI hierarchy through mcpbridge, one shared daemon for many hosts
v1.0	Stable protocol and embedding API, held by a check; PyPI, npm and the MCP Registry; a cursor that stays while the agent works
v1.2 (now)	SimMirror's own native helper instead of idb_companion, screens read from their pixels and settling past endless animations, an optional in-app debug SDK, and a Homebrew tap
v2.0	Real iPhones: view, install and launch without signing; full control through WebDriverAgent
Android	Android emulator support with the same viewer, cursor and tools
Website & launch	Landing page, video tutorials and guides

See ROADMAP.md and the milestones. How SimMirror relates to other tools is in Comparison.

Contributing

Contributions are welcome. Read CONTRIBUTING.md and the Code of Conduct; AI coding agents read AGENTS.md. Report security issues privately as described in SECURITY.md. Questions go to Discussions.

License

Apache-2.0. See NOTICE and THIRD_PARTY_LICENSES.md.

SimMirror is an independent project, not affiliated with Apple or Anthropic. iOS, iPhone, Xcode and Simulator are trademarks of Apple Inc.; Claude and Claude Code are trademarks of Anthropic PBC.