envoyproxy/ai-gateway ? reverse-engineered prompt
Reverse engineered prompt
Build me an open source AI gateway that sits in front of different generative AI providers and routes app traffic through a single entry point.
I want it to use Envoy Gateway style infrastructure, with a two tier setup where one layer handles authentication, top level routing, and global rate limiting, and the second layer handles access to self hosted model serving with smarter request routing for inference. It should support common providers like OpenAI, Azure OpenAI, Gemini, Vertex, Bedrock, Anthropic, Mistral, Cohere, Groq, Together, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, and a router service too.
Please make it feel production ready, with Kubernetes manifests, examples, and clear docs so someone can try a quickstart and understand how the pieces fit together. If you need to look up current docs online, go ahead. I mainly want a clean working project that shows how to manage unified access to AI services through Envoy.
Are you gonna build this?
make sure you review the code using coderabbit