hugoargerio-glitch/MacroFlow-Lakehouse ? reverse-engineered prompt

Build me an end to end data pipeline for macroeconomic and FX data.

It should pull data from the Central Bank of Brazil and Yahoo Finance, validate the incoming responses so bad records get caught early, and then land the raw data as dated JSON files in a bronze layer. After that, clean and standardize the data in a silver layer, fill gaps for missing market days, remove duplicates, and keep everything idempotent so reruns do not create bad copies. Then create a gold layer with useful analytics like moving averages, rolling volatility, spreads, and simple correlation style metrics.

Orchestrate the whole thing with Airflow, make it runnable locally with Docker, and include tests for the schema checks and transformation logic. Also set it up so the final analytical tables and views can be published to Snowflake. If you need to look up current docs online while wiring it together, go ahead.