microsoft/OmniParser ? reverse-engineered prompt

Build me a tool that can look at a screenshot of a computer screen and break it into the important parts, like buttons, icons, text, and other UI elements, then show what each part probably does and where it is on the screen.

I want to be able to run a simple demo notebook and a small web demo where I can upload a screenshot and see the parsed result overlaid on the image, with labels and clickable regions highlighted. It should feel useful for a vision based agent, so the output should make it easier to tell a model what to click or type next.

Please make it work well on common app screens, keep the setup straightforward, and include a few example images or a demo flow so I can quickly try it out. If you need current model or dependency details, look up the latest docs online and use the recommended weights or setup.