ManuelAF/webarchivecrawler ? reverse-engineered prompt

Reverse engineered prompt

Build me a simple web archive crawler tool that can start from a URL, follow links, and save what it finds so I can inspect archived pages later.

I want it to be easy to run from the command line, take a starting site and maybe a few basic options like how deep to crawl or where to store the results, and then quietly collect pages without crashing if some links fail. It should feel useful for grabbing a snapshot of a site and keeping track of the pages it discovered.

If there are any current best practices or library updates that would help with crawling and fetching pages, look them up online and use the safest approach. Keep it small and straightforward, since this is meant to be a practical utility rather than a big app.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab