peekaboo

Peekaboo — drive, observe, and critique any macOS app

**The job is not "is this clickable". The job is "does it work, and does

it look right". Every screenshot deserves a critique pass.**

Peekaboo is a single Swift CLI that combines screenshot, AX-tree inspection, synthetic input, and change-aware video capture against macOS apps. It's the macOS-native equivalent of Playwright + visual-regression review.

You invoke peekaboo directly — it's a full toolkit, not a fixed menu of recipes. Build orchestration belongs to your project's task runner (Justfile, Makefile, xcodebuild, etc.); UI driving is your domain.

No external AI keys. Ever. You are the LLM. When you need to "look at" the screenshot, view the PNG yourself. Skip peekaboo agent, peekaboo see --analyze, peekaboo image --analyze, peekaboo analyze — they all require an external API key.

When to invoke this skill

• ✅ User says "see what it looks like", "show me the UI", "verify it works",

"click X", "type Y", "take a screenshot", "drive the app", "test it end-to-end"

• ✅ After modifying any SwiftUI/AppKit view — verify it renders, works, and

still looks intentional

• ✅ When debugging a UI bug — capture before/after + critique each
• ✅ Before claiming a UI feature done in autopilot mode
• ❌ Skip if the change is purely backend / non-UI

Prerequisites

peekaboo --version                           # ≥ 3.0
peekaboo list permissions --json             # both must be granted

If peekaboo is not installed:

brew install steipete/tap/peekaboo

Then grant Screen Recording AND Accessibility to your terminal in System Settings → Privacy & Security. Both are required.

The loop (copy this checklist into your response and tick items off)

• [ ] Build the app — use the project's task runner
• [ ] Drive — get the app to the state under test
• [ ] Snapshot — peekaboo see --json --annotate --path /tmp/k.png
• [ ] CRITIQUE — view /tmp/k_annotated.png inline; score it against the

rubric. Optionally chain a design skill (critique/polish/layout)

• [ ] Fix any findings
• [ ] Re-verify — repeat snapshot + critique; diff against previous PNG

Discovering the target app

Before running anything, you need the bundle ID (preferred) or app name:

# Find a running app's bundle ID
peekaboo list apps --json | jq '.data.applications[]|select(.name|test("YourApp";"i"))|{name,bundleIdentifier,processIdentifier}'

# Always prefer --bundle-id over --app <DisplayName>:
#   - bundle ID is stable (e.g. com.example.myapp)
#   - the running process registers as CFBundleName, which often differs
#     from the Xcode scheme. Using the scheme name fails APP_NOT_FOUND.

Cache the bundle ID and any display-name caveats in your project's AGENTS.md / CLAUDE.md so future runs don't re-discover them.

Terminology (one-to-one with peekaboo's command names)

| Word | Means | Command | |----------------|----------------------------------------------------------|--------------------------| | Snapshot | Annotated PNG plus AX-tree JSON (the "DOM+screenshot") | peekaboo see | | Screenshot | Raw PNG only, no AX tree | peekaboo image | | Recording | Change-aware MP4 video | peekaboo capture live |

Default to snapshot (peekaboo see) — the AX JSON lets the next step click by element ID without re-parsing the screen.

Step 1 — Drive the UI

The full driving surface (run peekaboo <cmd> --help for details, or peekaboo learn for the comprehensive guide):

| Need | Command | |---------------------------------------|----------------------------------------------------------| | Launch / quit app | peekaboo app launch --bundle-id $BID --wait-until-ready<br>peekaboo app quit --app $BID | | Inspect UI (annotated PNG + JSON) | peekaboo see --app $BID --json --annotate --path /tmp/k.png > /tmp/k.json | | Click by AX identifier (most robust) | See "Click by .accessibilityIdentifier" recipe below | | Click by visible label | peekaboo click "Settings" --app $BID | | Click by element ID from see | peekaboo click --on elem_42 --snapshot $SID --app $BID | | Type text | peekaboo type "hello" --app $BID<br>(append --return to press Enter) | | Single key | peekaboo press return / peekaboo press escape | | Hotkey | peekaboo hotkey --keys "cmd,b" --app $BID | | Menu bar | peekaboo menu list --app $BID (discover)<br>peekaboo menu click --app $BID --path "View > Toggle Sidebar" | | System file/save dialog | peekaboo dialog list --app $BID<br>peekaboo dialog click --button "Open" --app $BID<br>peekaboo dialog input --text "..." --field "..." --app $BID<br>peekaboo dialog file --path "/Users/me" --select "Open" --app $BID | | Scroll inside a list / pane | peekaboo scroll --direction down --amount 5 --on elem_N | | Drag-and-drop | peekaboo drag --from elem_A --to elem_B --app $BID | | Resize/reposition window | peekaboo window set-bounds --app $BID --x 0 --y 0 --width 1024 --height 640 | | Read/write clipboard | peekaboo clipboard get / peekaboo clipboard set --text "x" | | Paste from clipboard | peekaboo paste --app $BID | | Open URL / deep link | peekaboo open "x-myapp://..." | | Replay a saved scenario | peekaboo run scripts/onboarding.peekaboo.json |

Throughout this skill, $BID is the target app's bundle ID (com.example.myapp).

Click by .accessibilityIdentifier — the killer pattern

If your SwiftUI / AppKit code tags interactive views with .accessibilityIdentifier(...), Peekaboo surfaces those exact strings as ui_elements[].identifier. Queries are then immune to copy changes:

peekaboo see --app $BID --json --annotate --path /tmp/k.png > /tmp/k.json

SID=$(jq -r .data.snapshot_id /tmp/k.json)
ID=$(jq -r '.data.ui_elements[]|select(.identifier=="header.utility.settings").id' /tmp/k.json)

peekaboo click --on "$ID" --snapshot "$SID" --app $BID

If you reach for a copy-based click (peekaboo click "Some Label") more than once on the same control, add a stable identifier in your project's chosen namespace (e.g. header.tab.sessions, panel.settings.save, row.<id>.delete). Both peekaboo queries and any XCUITest become resilient to copy changes:

Button("Settings") { … }
    .accessibilityIdentifier("header.utility.settings")

Document the namespace in your project's AGENTS.md so additions stay consistent.

Step 2 — Capture (anatomy of peekaboo see)

peekaboo see --app $BID --json --annotate --path /tmp/k.png > /tmp/k.json

This writes two PNGs:

• /tmp/k.png — raw screenshot
• /tmp/k_annotated.png — same image overlaid with elem_N markers

(this is the one you usually want to view)

…and prints a JSON envelope {success, data, debug_logs} where data has:

| Field | Meaning | |------------------------|--------------------------------------------------------| | snapshot_id | UUID — pass to --snapshot for click stability | | screenshot_raw | path to /tmp/k.png | | screenshot_annotated | path to /tmp/k_annotated.png | | application_name | display name (CFBundleName)