inferedge-ai/evalkit ? reverse-engineered prompt

Reverse engineered prompt

Build me a self hosted app that helps people improve an AI agent from real runs.

I want a simple system where I can import traces or agent runs, turn bad runs into reviewable eval cases, and let a domain expert approve or comment on them in plain language. Then I want to run those cases against an agent, see the full step by step trajectory, scores, tool calls, timing, and cost, and compare one experiment against another so it is obvious what got better or worse.

It should also keep a history of cases and reviews, support workspaces for different people or teams, and let approved cases become the regression suite. If it helps, include a basic web UI plus a Python SDK or API for running experiments and sending traces. Make the setup something I can run locally first, and if you need to check current docs or libraries online, go ahead.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab