Nitinx12/Walmart_DBT ? reverse-engineered prompt

Reverse engineered prompt

Build me a Walmart data pipeline project that takes product and sales data from MongoDB, loads it into PostgreSQL, and organizes it into bronze, silver, and gold layers. I want the raw data copied in first, then cleaned up and deduplicated, then turned into a nice analytics model for reporting on brands and categories.

It should run as a real pipeline, not just a notebook. Use Airflow to orchestrate the steps, use PySpark for the extract and load part, and use dbt for the transformations and tests. Please make it work in Docker too, so I can bring up the whole stack locally and run the pipeline from there. Add data quality checks so each stage only continues if the previous one looks good. If anything needs current setup details, look up the latest docs online.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab