siddzzzz/Tarsier ? reverse-engineered prompt

Build me a Python tool that lets an LLM control desktop and web apps by reading accessibility trees instead of screenshots.

I want it to work on Windows, macOS, and Linux, and to capture the active app, clean up the raw accessibility data, and turn it into a compact YAML snapshot that looks like a simple semantic page tree. Then it should let an agent find elements by their role and label, like buttons, text fields, menus, and links, and perform basic actions such as click, type, scroll, and focus in a reliable way.

Please include a small MCP server so other agents can read the current UI state and send actions back. Also add a few example scripts showing it working with both a desktop app and a web page. Keep it lightweight, fast, and focused on deterministic interaction, not screenshots or vision. If you need to check current library docs for accessibility APIs, feel free to look them up online.