yijunyu/jev-rs ? reverse-engineered prompt

Reverse engineered prompt

Build me a Rust tool called jev that can ask an LLM simple judgment questions about some state and return probabilities instead of free text.

I want it to work in three ways. First, as a command line tool where I can send in a piece of text and ask yes or no, pick one option from several choices, or rate something on a small ordered scale. Second, as a local HTTP server that exposes a simple API for /v1/systemone and works with TypeSafe style clients. Third, as an MCP tool for coding agents so they can call a judge tool over stdio when they need classification or triage.

It should be able to talk to a local model server or an OpenAI compatible backend, read log probabilities, and turn that into confidence scores. Include commands for trying a single request, evaluating a dataset, and calibrating probabilities so the output is more trustworthy. Make it easy to install and run locally, and keep the whole thing focused on fast, structured judgments rather than generated text.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab