akitaonrails/llm-coding-benchmark ? reverse-engineered prompt
Reverse engineered prompt
Build me a small Python project that benchmarks coding models by giving them the same Rails app brief and then collecting the results in a clean, repeatable way.
I want to be able to run a set of model tests, save each run in its own folder, keep the raw agent logs, and also generate a normalized result file and a Markdown summary. The main flow should use OpenCode to run the agent, first to build the app, then to continue the same session and check that the app boots locally and works with Docker and docker compose.
Please make it easy to compare local Ollama models and cloud models under the same prompt family, and include simple docs for how the benchmark works and how the scores are reviewed. If there are helper scripts, make them practical and easy to rerun. Use current docs online if you need to.
Are you gonna build this?
make sure you review the code using coderabbit