agentscope-ai/PawBench ? reverse-engineered prompt

Reverse engineered prompt

Build me a Python project called PawBench that helps compare how well different LLMs do when they are run inside different agent harnesses. I want it to run a set of benchmark tasks, collect the results, and show a leaderboard that compares model and harness combinations side by side.

It should support running single tasks or groups of tasks, saving outputs and traces, and exporting results in a format that can be used to build a simple website with a leaderboard, task pages, and slice views so I can see where a model or harness is doing well or failing. Please make it work with Docker for the harness setup, and include a clean command line way to start benchmark runs and then view or rebuild the results.

If you need to look up current docs for anything, go ahead and check online. Keep the structure easy to understand, and make sure it feels like a real benchmark project rather than just a demo.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab