comet-ml/opik ? reverse-engineered prompt

Reverse engineered prompt

I need to build an open-source platform for LLM observability and evaluation. This tool should help me debug, test, and monitor my AI applications from prototype to production.

I want to be able to see detailed traces of all the LLM calls my application makes, log entire conversations, and understand the workflow of my RAG systems and agents. It needs a strong evaluation component, allowing me to use an "LLM as a judge" to check for things like hallucinations or see how relevant an answer is. I also want to manage datasets for testing and run experiments to compare different prompts or models.

The platform should have a production-ready dashboard to monitor performance, token usage, and user feedback over time. It would also be great to have a "playground" area for experimenting with prompts. Essentially, I need a comprehensive system with a server and a Python SDK to help me build better and safer AI apps.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab