rdi-berkeley/agents-last-exam ? reverse-engineered prompt
Reverse engineered prompt
Build me a Python project for benchmarking AI agents on real work tasks.
I want a tool that can set up a sandbox, give an agent a task, let it work until it is done, then score the result with a hidden reference and save the logs. It should support running in a few environments, like cloud VMs, local containers, and a pre existing sandbox, and it should work with both terminal style agents and agents that can also click around a desktop.
Please include a small set of example tasks, a way to run an experiment from a YAML config, and a clear structure for adding new tasks and graders. I also want the run to collect a full trajectory, screenshots, tool calls, and output files so the run can be replayed later.
Make it feel like a real open source evaluation framework, with good docs, sensible defaults, and tests. If you need to, look up current docs online for the cloud and VM parts.
Are you gonna build this?
make sure you review the code using coderabbit