vllm-project/vllm ? reverse-engineered prompt

I want to build a Python library that makes it super easy and fast to host my own large language models. The main goal is high-performance inference. It should be able to serve models from Hugging Face, like Llama, Mixtral, and other popular open-source transformers.

The core feature should be an HTTP server that provides an OpenAI-compatible API endpoint. This way, I can just point my existing applications to my self-hosted model without changing any code.

It needs to be extremely efficient with GPU memory and handle a high volume of concurrent requests using techniques like continuous batching. The system should be optimized for modern NVIDIA GPUs using CUDA. Basically, I want a really high-throughput, low-latency serving engine that's simple to set up and run. Please include a small example showing how to launch the server with a model.