agentscope-ai/OpenJudge ? reverse-engineered prompt

Reverse engineered prompt

Build me an open source Python tool for evaluating AI apps like chatbots and agents.

I want to be able to paste in test cases, pick or create grading rules, run evaluations over a batch of outputs, and then see clear results that show what went well and what needs work. It should include a set of ready made graders for common things like relevance, correctness, JSON structure, tool use, memory, and overall response quality, plus a way to generate custom rubrics from a short description when I do not already have one.

Please also make a simple web app where I can try graders in the browser, inspect scores, and compare runs without needing to set up much. If it makes sense, let the grading results be turned into reward signals too, so this can help with improving and fine tuning an agent later. Keep it easy to run locally, well organized, and add enough docs and tests so someone can understand how to use it. If you need current docs or examples, look them up online first.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab