huggingface/text-embeddings-inference ? reverse-engineered prompt
Reverse engineered prompt
Build me a fast local service for serving text embedding models, with a simple API that I can call from my apps to turn text into vectors.
I want it to support a few common open source models from Hugging Face, run well in Docker, and start up quickly without a lot of setup. It should handle batching so it can process multiple requests efficiently, and it should be able to work with GPUs when they are available, but still run locally on a Mac too. Please include a clean HTTP API, and if it makes sense, gRPC as well.
It would be great if it can also do reranking or basic sequence classification for supported models, plus give me metrics and tracing so it feels production ready. Make sure it can load models from safetensors or ONNX, and include clear instructions for running it with a public model, a private model, and an air gapped setup. If you need current Hugging Face docs or model details, look them up online.
Are you gonna build this?
make sure you review the code using coderabbit