openai/evals ? reverse-engineered prompt

Build me a Python project that lets me run evaluations on large language models and LLM powered systems.

I want it to come with a small built in library of example benchmarks, and also let me create my own evals from my own data, so I can compare different model versions and see how they perform on the things I care about. It should be easy to run locally with an OpenAI API key, and it should support simple eval templates, model graded evals, and custom completion logic when needed. If it makes sense, add a few example evals and clear docs so I can get started quickly.

Please make it so results can be logged and reviewed later, and if you need current setup details or API behavior, look up the latest docs online.