Coordinate-based UI tests break. So we read the accessibility tree instead — from inside the simulator.

Every recorded mobile test I have ever inherited died the same way: someone moved a button. The recording said "tap at (340, 712)". The redesign moved that button up by one row, and the test kept tapping — now on empty space, or whatever happened to land there instead. It didn't fail right away. Three sprints later, it started failing in confusing ways, and by then nobody trusted the suite anymore. The fix isn't a better recorder. It's recording a different thing: not where you tapped, but what you tapped. That needs an element tree, and for a while we didn't have one. tapflow is an open-source, self-hosted tool that streams iOS simulators and Android emulators into a browser, so a whole team can test builds without installing anything. Until now, everything it moved was pixels in one direction and taps in the other. This post is about getting an element tree out of a simulator with no window, on both platforms. What we do with that tree — replaying flows that survive a redesign — is the next post in this series. The automation axis this feeds — the flow runner and the MCP server — is experimental . The manual browser QA path is the mature one. The constraint: no WebDriverAgent, and no simulator window tapflow already injects touches into the iOS simulator without WebDriverAgent — it loads CoreSimulator.framework and pushes HID events through SimDeviceLegacyHIDClient (that story is ep.1 ). Streaming reads the framebuffer IOSurface directly. Neither path needs Simulator.app on screen, and that's deliberate: an agent Mac in a closet running four simulators shouldn't be babysitting four windows. So whatever we used for the tree had to follow the same rule. No WDA to install and keep in sync with Xcode. No simulator window on screen. Our first attempt ran into exactly that limitation. macOS exposes an accessibility API ( AXUIElement ), and Simulator.app publishes its content through it. We wrote a helper around it, and it worked perfectly on a developer's laptop. On the