viggy28/streambed — reverse-engineered prompt

Reverse engineered prompt

GitHub

Build me a Go command line tool called Streambed that lets someone stream changes from a Postgres database into Apache Iceberg tables stored on S3, so analytics can run without hitting the production database.

It should connect to Postgres using logical replication, read inserts, updates, and deletes from the WAL, write the data out as Parquet files, and keep Iceberg metadata up to date. It should also have a simple query server that speaks the Postgres protocol, so I can connect with psql or another Postgres client and query the mirrored tables.

Include commands for running the main sync process, doing a one time backfill of a table, starting only the query server against existing Iceberg data, and cleaning up a table’s stored data. Make it easy to run locally with Docker using Postgres and MinIO, and document the basic build, sync, and query flow in the README.

Want more depth? Deep Reverse