jina-ai/reader ? reverse-engineered prompt
Reverse engineered prompt
Build me a web service that acts as a 'reader' and 'search' API to prepare content for an LLM.
The main feature should be a 'reader' endpoint. I want to give it any URL, and it should return the clean, main content of that page in a simple text or markdown format, stripping away all the ads, navigation, and other clutter. This needs to work on complex, modern websites that rely on JavaScript, and it should even be able to extract text from a PDF if I give it a link to one.
The second feature is a 'search' endpoint. Instead of just a URL, I'll give it a search query. It should then use a search engine to find the top 5 or so results, and then automatically run the 'reader' process on each of those links. The final output should be the full, cleaned-up content from all those top pages.
Let's build this with TypeScript and Node.js. It should be able to handle complex pages, maybe using a headless browser behind the scenes.
Are you gonna build this?
make sure you review the code using coderabbit