airscholar/e2e-data-engineering ? reverse-engineered prompt

Build me an end to end data engineering project that pulls random user data from the randomuser.me API, loads it into a database, streams it through Kafka, processes it with Spark, and stores the final results in Cassandra.

I want the whole thing to run in Docker with a simple docker compose setup, so I can spin everything up locally without a lot of manual setup. Please include Airflow to orchestrate the pipeline, PostgreSQL for the initial storage, Kafka and Zookeeper for streaming, Spark for processing, and Cassandra for the output. It should feel like a complete working demo of a real time data pipeline, not just a skeleton.

If you need to check current docs or best practices for any of the services, go ahead and look them up online. Make it easy to follow, with clear code and anything needed to run the full flow from ingestion to storage.