sgl-project/sglang ? reverse-engineered prompt

I want to build a high-performance serving framework in Python for large language models. The main goal is speed and efficiency—it should deliver very low latency and high throughput for inference. It needs to be able to serve not just standard LLMs like Llama, Mistral, and Gemma, but also multimodal models for image and video generation.

The framework should be built with broad hardware support in mind, capable of running on a single NVIDIA or AMD GPU but also scaling up to large, distributed clusters. Please implement advanced performance optimizations like prefix caching with RadixAttention, continuous batching, and various forms of parallelism. It should also support quantization and be compatible with models from Hugging Face.

Finally, expose the models through an OpenAI-compatible API endpoint. Include documentation and examples to show how to use it with different kinds of models. Look up current best practices for LLM inference optimization to make sure it's as fast as possible.