adityagupta000/ModelMesh ? reverse-engineered prompt

Build me a multi model inference gateway that lets me add and manage OCR and speech to text models from an API instead of hardcoding routes.

I want a FastAPI based service with login and API key auth, a model registry in PostgreSQL, Redis for rate limiting and job queues, and worker services that can process requests asynchronously. It should support both normal HTTP requests and real time streaming over WebSockets, with gRPC for internal model calls when needed.

Please make model routing data driven, so I can add a model, change its version, or switch its protocol without redeploying the gateway. I also want basic canary routing, request polling for async jobs, retries for failed jobs, and some observability with metrics and request logs. If it makes sense, set it up so it can run locally with Docker Compose and also be deployed on Kubernetes. Use the current docs online if you need to fill in any gaps.