castorini/rank_llm — reverse-engineered prompt
Reverse engineered prompt
Build me a Python toolkit for reranking search results with language models, mainly for research use, but make it easy to run from the command line too. I want it to handle pointwise, pairwise, and especially listwise reranking, with support for local open source models as well as hosted providers like OpenAI, Gemini, and OpenRouter. It should be able to start from retrieved candidates, rerank the top results for standard datasets, save outputs, and run evaluation so I can compare models in a reproducible way.
Please include a clean command line flow for rerank, prompt listing, viewing results, evaluation, and serving over HTTP or MCP. I would also like simple Python examples for retrieval, reranking, and analysis, plus support for custom prompt templates from YAML. Keep the install story straightforward, with optional extras for local inference, cloud providers, API serving, and training. If anything is unclear, look up the current docs online and make sensible choices that match the repo.
Want more depth? Deep Reverse