MCP Tools Reference

Haptix exposes a stable MCP tool surface through the protocol. Every tool listed here is an MCP tool — called by your AI agent, not by you directly. Each one has a haptix CLI equivalent (the command names differ slightly); see the CLI reference.

There are exactly 34 MCP tools.

Tool index

Session

start_session

Begin an agent session. This is the entry point — call it before any other tool.

start_session auto-discovers connected USB devices and auto-selects if only one is found. It returns the device name, model, screen size, and connection state. You don't need to call list_devices → select_device → device_info separately — start_session handles discovery for you.

Returns a session code, device info (if a device is connected), and confirmation.

end_session

End the active session. Removes the activity halo, stops recording, and returns a summary with duration and tool call count. Always call when done.

No parameters.

Device

device_status

Check connection state. Returns connection and session status.

No parameters.

device_info

Get device details: model, screen size, OS version, orientation, battery level, app bundle ID, debugger state, and UI automation status.

No parameters.

Returns JSON with: model, osVersion, screenWidth, screenHeight, scaleFactor, orientation, batteryLevel, isCharging, bundleIdentifier, appName, debuggerAttached, uiAutomationEnabled.

list_devices

List all connected USB devices. Returns device name, app name, bundle ID, model, and connection status for each.

No parameters.

Most agents don't need this — start_session auto-discovers and auto-selects devices. Use list_devices only when you have multiple devices and need to choose.

select_device

Bind this session to a specific device. All subsequent commands target this device. If not called, commands target the first available device.

launch_app

Launch an app by bundle identifier.

activate_app

Bring an already-installed app to the foreground.

terminate_app

Terminate a running app.

press_home

Press the Home button, returning to the home screen.

No parameters.

press_volume_up

Press the volume-up hardware button.

No parameters.

press_volume_down

Press the volume-down hardware button.

No parameters.

installed_apps

List installed apps on the device. Returns app name and bundle ID for each.

keyboard_state

Report whether the on-screen keyboard is currently visible and the screen region it covers. When the keyboard is up, the agent uses this to refuse interacting with controls hidden behind it instead of silently missing them.

No parameters.

Returns whether the keyboard is visible and, when it is, the covered region.

Screenshot & Observation

screenshot

Capture the screen. The primary tool for visual discovery.

Screenshots are captured Mac-side from the device's video feed. Haptix waits for a fresh frame from the selected device and fails clearly if it cannot get one. There is no device-side screenshot fallback, and Haptix refuses to guess across attached phones if the capture source drifts.

Annotated screenshots show green boxes for interactive elements and blue boxes for static elements. The overlay is composed Mac-side on the captured frame, so annotated inspection works on any iOS app — including apps the developer doesn't own. No SDK embed required. (The live, on-device annotate tool further down is a separate path that does still require HaptixKit.)

accessibility_tree

Get UI elements with labels, identifiers, values, traits, and coordinates.

Compact mode returns a flat list of meaningful elements. Full mode returns the complete nested view hierarchy. Prefer compact — it's far more token-efficient.

Reads are depth-bounded for speed on very large or deeply nested screens — a default max_depth of 2 keeps infinite-scroll feeds responsive where a full walk used to stall. Raise max_depth up to 8 when you need deeper parent-child structure.

find_element

Locate a single element by type and/or label without walking the full tree. Returns the matching element's identifier, frame, label, and traits.

Pass at least one of type or label. Useful when you already know what you're looking for and want a fast, targeted lookup instead of a full accessibility_tree.

Gestures

All gesture tools return a multi-content response: a fresh follow-up screenshot of the resulting UI state, confirmation text, and any console output captured during the action. No need to call screenshot after every gesture unless you want a different format or a higher-resolution capture.

tap

Single tap. Prefer identifier or label over coordinates — they're more reliable and survive layout changes.

Includes hit feedback showing which element was tapped: [Hit] "Submit" [button] identifier: "submitButton".

double_tap

Double tap. Same parameters as tap.

long_press

Long press. Triggers context menus. Specify duration for custom hold time.

swipe

Swipe between two points. Use on background areas to scroll pages. Use on picker wheels to change values. Use on sliders to move the thumb.

drag

Drag between two points (slower than swipe, with press-and-hold). Use for reordering lists, moving items, or dragging handles. For sliders, prefer swipe.

draw_path

Draw one continuous one-finger stroke through an ordered list of waypoints. Use it for Freeform drawing, signatures, circles, lasso paths, and organic drags where breaking the motion into separate straight-line calls would incorrectly lift the finger.

Separate draw_path calls imply lifting the finger on purpose.

Long paths (many waypoints, multi-second strokes) emit MCP notifications/progress while the stroke runs. Clients that subscribe to progress see in-flight state — useful for keeping users oriented during a slow draw. Clients that don't subscribe simply wait for the final response; the tool still completes correctly.

scroll

Scroll page content in a direction. Simpler than swipe for basic navigation — no coordinates needed.

pinch

Pinch gesture at a center point. scale less than 1 pinches in (zoom out), greater than 1 pinches out (zoom in).

rotate

Two-finger rotation at a center point.

Text Input

type_text

Type text into the focused input field. Tap the field first to focus it.

clear_text

Clear all text in the focused input field.

No parameters.

Annotation

On-device annotation tools draw directly on the device screen. These use the live, on-device path and require HaptixKit (distinct from the Mac-side overlay in annotated screenshot).

annotate

Draw an annotation on the device screen.

clear_annotations

Remove all on-device annotations.

No parameters.

Console

get_console_output

Get app console output including print(), NSLog(), os_log(), errors, and warnings. Console output is also auto-included in every gesture response, so you rarely need to call this directly.

Notes

Notes persist short observations across an agent's run so it can recall context later without re-deriving it from the screen.

store_note

Save a note for later recall.

recall_notes

Retrieve stored notes.

consolidate_notes

Merge and compact stored notes into a tighter set, reducing redundancy.

No parameters.

Key behaviors

• Auto-screenshot: Every gesture tool returns a screenshot of the resulting UI state. No need to call screenshot after each action.
• Fresh-or-fail screenshots: Haptix waits for a fresh Mac-side frame from the selected device. If the capture source drifts across attached phones, it returns a clear error instead of another device's pixels.
• Keyboard awareness: When the on-screen keyboard is up, the agent uses keyboard_state to avoid acting on controls hidden behind it rather than silently missing the target.
• Hit feedback: Tap tools include which element was hit: [Hit] "Submit" [button] identifier: "submitButton" -- base layer.
• Auto-console: Console output captured during the gesture is included in every response.
• Coordinates: All coordinates are in screen points (not pixels), origin (0, 0) at top-left. Read them from annotated screenshots — don't estimate from image pixels.
• Settle delay: After each gesture, the server waits 500ms for the UI to settle before capturing the follow-up screenshot.
• Auto-discovery: start_session discovers USB devices automatically. No need for list_devices → select_device unless you have multiple devices.