m92vyas/llm-reader ? reverse-engineered prompt
Reverse engineered prompt
Build me a small Python library that takes any webpage and turns it into clean text that an LLM can actually use well.
I want it to help with scraping and RAG, especially for pages that have product lists, images, tables, and lots of links. It should be able to take raw HTML or a URL, clean up the page, and return an LLM friendly version of the content that makes it easier to extract things like product names, prices, image links, and page links.
Please make it simple to use from Python, with async support so it can work on multiple pages at once, and use a modern browser based approach for getting page source when needed. Add a few easy examples in the README showing how to fetch a page, process it, and pass the result into an LLM prompt.
If you can, include sensible defaults, good error handling, and a clear way to plug in my own scraping setup or proxy service if I already have one.