psf/requests-html ? reverse-engineered prompt
Reverse engineered prompt
Build me a Python library for scraping websites that feels as easy as using Requests, but with built in HTML parsing.
I want to be able to make normal GET requests, keep cookies and redirects working, and then quickly pull data from the page with CSS selectors, XPath, and simple helpers for links and absolute links. It should also handle pages that need JavaScript, so if needed it can load the page in a browser behind the scenes and still let me inspect the rendered HTML.
Please include both normal and async support, since I want to fetch a few pages at once. I’d also like a clean way to export page content or elements to markdown. Keep the API simple and friendly, like something a beginner could use without much setup.
If you need to check current docs or best practices for the browser automation part, look them up online.
Are you gonna build this?
make sure you review the code using coderabbit