bytedance/UI-TARS ? reverse-engineered prompt

Build me a working Python demo for UI TARS that can take a screenshot, send it to the UI TARS model, read the model’s Thought and Action response, turn that action into pyautogui code, and optionally run it on my computer.

I want this to feel like a simple local assistant for GUI automation. It should let me type a task like “open the browser and search for weather”, choose desktop mode, show the screenshot being used, show the model response, show the parsed action, and have a safe confirm button before actually moving the mouse or typing.

Use the existing UI TARS parsing tools and prompt templates from this repo. Include clear setup steps for the model endpoint or Hugging Face deployment, and make the app easy to run locally. Keep it simple and safe, with logging and error messages when coordinates or actions can’t be parsed. Look up the current docs online if you need to.