damklis/DataEngineeringProject ? reverse-engineered prompt
Reverse engineered prompt
Build me an end to end news pipeline app that pulls the latest stories from RSS feeds, runs on a schedule, and makes the results easy to browse through an API.
I want it to scrape and validate incoming news, send the records through a streaming pipeline, store them in MongoDB, keep Elasticsearch in sync for search, and also save copies in object storage for durability. Please set it up so I can run the whole stack locally with Docker and docker compose, and include a simple API where I can list all news and search by text, title, or language.
If anything about the feed setup or search behavior is unclear, use the README as the guide and look up current docs online if you need to. Also make sure there is a straightforward way to start, stop, and test everything from the project root.
Are you gonna build this?
make sure you review the code using coderabbit