Compatibility
What works, what doesn't, and what we're fixing. Last updated February 2026.
Status key
| Status | Meaning |
|---|---|
| Works | Reliable. Agents can use this confidently. |
| Partial | Works in some cases, has known edge cases or workarounds. |
| Broken | Does not work. Actively being fixed. |
Gestures
| Gesture | Status | Notes |
|---|---|---|
| Tap (coordinates) | Works | Multi-strategy: accessibility → control → probe → system event |
| Tap (label) | Works | Finds element by accessibility label, taps its center |
| Tap (identifier) | Works | Finds element by accessibility identifier, taps its center |
| Double tap | Works | Two touch sequences with correct tap count |
| Long press | Works | System-backed touch synthesis. Opens context menus |
| Scroll (directional) | Works | scroll tool with direction + amount. Programmatic content offset |
| Swipe (start→end) | Partial | Shortcut interception can cause issues on pickers and custom controls. See Swipe Issues |
| Drag (start→end) | Broken | Missing dwell phase and system-backed touches. See Drag Issues |
| Pinch | Broken | Gesture recognizers reject synthetic multi-touch. See Pinch and Rotate |
| Rotate | Broken | Same issue as pinch. See Pinch and Rotate |
| Type text | Works | Uses insertText() on focused field |
| Clear text | Works | Clears all text in focused input |
Controls
| Control | Status | Tool | Notes |
|---|---|---|---|
| UIButton / SwiftUI Button | Works | tap |
Accessibility activation with fallbacks |
| UISwitch / SwiftUI Toggle | Works | tap |
Accessibility activate toggles the switch |
| UISlider (UIKit) | Works | swipe |
Programmatic setValue + value changed event |
| SwiftUI Slider | Works | swipe |
Accessibility increment/decrement (0–100 percentage) |
| UITextField / UITextView | Works | tap → type_text |
Tap to focus, then type |
| SwiftUI TextField | Works | tap → type_text |
Same as UIKit |
| Tab bar buttons | Works | tap |
Standard tap on tab items |
| Navigation bar buttons | Works | tap |
Back buttons, bar button items |
| UIPickerView (wheel) | Works | swipe |
Programmatic row selection |
| UIDatePicker | Works | swipe |
Programmatic date setting |
SwiftUI Picker (.wheel) |
Works | swipe |
Uses wheel control internally |
SwiftUI Picker (.menu) |
Broken | — | Uses context menu internally. See Menu Pickers |
SwiftUI Picker (.inline / .segmented) |
Works | tap |
Tap on the option directly |
| ColorPicker | Works | tap |
Opens color picker sheet |
| Context menus | Broken | — | Long press opens, but tapping items fails. See Context Menus |
| Pull-down menus (UIMenu) | Broken | — | Same root cause as context menus |
UIAlertController (.alert) |
Partial | tap |
Alert buttons sometimes respond, inconsistent |
UIAlertController (.actionSheet) |
Partial | tap |
Same as alerts |
| Share sheet | Broken | — | System-presented, synthetic touches rejected |
SwiftUI .sheet / .fullScreenCover |
Works | tap |
App-managed sheets work normally |
SwiftUI .confirmationDialog |
Broken | — | Uses system action sheet internally |
| Drag and drop | Broken | — | See Drag and Drop |
| Reorderable lists | Broken | — | Requires long-press-then-drag |
Observation & Inspection
| Capability | Status | Notes |
|---|---|---|
| Screenshot (app mode) | Works | Clear capture of app content |
| Screenshot (full mode) | Works | Includes status bar |
| Screenshot (region mode) | Works | Specific area capture |
| Annotated screenshots | Works | Bounding boxes with labels, identifiers, coordinates |
| Filtered annotations | Works | Filter by: interactive, adjustable, button, text, input, image |
| Element highlight | Works | Yellow box + crosshair on single element |
| Screenshot streaming | Works | 2–10 fps continuous capture. Requires USB — connect your device with a cable to unlock the live screen stream |
| Accessibility tree (compact) | Works | Flat list, ~2K tokens |
| Accessibility tree (full) | Works | Nested hierarchy, ~20K tokens |
| Console output (stdout) | Works | print(), debugPrint(), dump() |
| Console output (stderr) | Works | NSLog(), framework warnings |
| Console output (os_log) | Works | Logger / os_log with level filtering |
| Auto-console on gestures | Works | Included in every gesture response |
| Device info | Works | Model, screen size, OS, battery, bundle ID |
| Device status | Works | Connection state, session status |
Connection & Setup
| Capability | Status | Notes |
|---|---|---|
| USB device discovery | Works | Automatic via devicectl |
| USB auto-connect | Works | Plug in and go |
| Activity indicator (halo) | Works | |
| Notes (store/recall) | Works |
Known issues — detail
Context menus
Affects: SwiftUI .contextMenu, UIContextMenuInteraction, any UIMenu-based presentation.
What happens: Long press successfully opens the context menu. Once the menu is open, tapping on menu items does nothing. The tap visually lands on the item but the selection is silently rejected.
Root cause: The context menu's internal gesture recognizer validates system-level backing on touch events. Haptix tap uses accessibility activation which lacks this backing, so the menu rejects the touches.
Workaround: None reliable. The menu uses a touch-down-hold-drag-release pattern where the item under the finger at release time gets selected. This requires predicting menu item positions before the menu appears.
Status: Fix in progress — extending system-backed touch synthesis to tap events inside context menu presentations.
Menu-style pickers
Affects: SwiftUI Picker with .menu pickerStyle (the default style in many contexts).
What happens: The picker element reports as enabled: false in the accessibility tree. Long press opens the dropdown, but selecting an option fails (same as context menus).
Root cause: .menu pickers use UIContextMenuInteraction internally. Same validation issue.
Status: Same fix path as context menus. Also investigating programmatic picker value selection via accessibility APIs.
System-presented sheets
Affects: System UI presented by iOS itself — alert controllers, action sheets, share sheets, color picker details, photo picker.
What happens: These sheets are presented by the system, not by your app. Tap synthesis may or may not work depending on the specific sheet. Alert buttons sometimes respond, share sheet buttons consistently fail.
Root cause: System-presented UI may run in a different process or use stricter touch validation.
Workaround: For alerts with known button text, try tapping by label. Success is inconsistent.
Swipe shortcut interception
Affects: Swipe gestures on picker wheels, custom scroll-driven controls, pull-to-refresh.
What happens: Vertical swipes are intercepted and redirected to programmatic content offset scrolling. This works for basic page scrolling but fails on controls that need real gesture events (picker wheels, parallax, sticky headers).
Root cause: The touch synthesizer has shortcut paths that intercept before touch events are sent: edge swipe → pop navigation, horizontal on slider → set value, vertical → set content offset.
Workaround: Use the scroll tool for page navigation (it's designed for content offset). For picker wheels, use swipe with small movements directly on the wheel column.
Drag and drop
Affects: Drag-and-drop interactions, reorderable lists, any control requiring long-press-then-drag.
What happens: The drag tool internally calls swipe, which means it inherits swipe shortcut issues. There is no dwell phase, so long-press-then-drag patterns never activate.
Root cause: Missing dwell phase at start point and no system-backed touch synthesis for continuous gestures.
Workaround: None currently.
Status: Implementing proper drag with system-backed touches and configurable dwell phase.
Pinch and rotate
Affects: UIPinchGestureRecognizer, UIRotationGestureRecognizer, SwiftUI MagnifyGesture, SwiftUI RotateGesture.
What happens: The tool completes without error, but the gesture recognizer never fires. The target view doesn't respond — scale/angle stays unchanged.
Root cause: Multi-touch gesture recognizers require two simultaneous touch objects with system-level backing. The current synthesis creates two touch points but without proper backing, the recognizers silently ignore them.
Workaround: None. Use a reset button or programmatic API instead.
Status: Extending system-backed touch synthesis to support multi-touch (two simultaneous finger events).
Summary
| Category | Working | Partial | Broken |
|---|---|---|---|
| Core gestures (tap, long press, type) | 7 | 0 | 2 |
| Standard controls (buttons, sliders, toggles, text fields) | 9 | 0 | 0 |
| Picker controls (wheel, date, segmented) | 4 | 0 | 0 |
| Menu-driven controls (context menu, menu picker, pull-down) | 0 | 0 | 3 |
| System-presented UI (alerts, action sheets, share sheets) | 0 | 2 | 2 |
| Continuous gestures (swipe, drag, scroll) | 1 | 2 | 1 |
| Observation (screenshots, tree, console) | 14 | 0 | 0 |
| Total | 35 | 4 | 8 |
The core interaction loop — see the screen, tap controls, type text, read console output — is solid. The gaps are in system-managed UI (menus, sheets) and advanced gesture patterns (drag-and-drop, multi-touch). All broken items share the same root cause: touch events that lack system-level backing being rejected by UIKit's internal validation.