maci0/agave ? reverse-engineered prompt

Build me a fast local LLM inference tool in Zig that can run modern open models from a downloaded GGUF or SafeTensors file. I want it to work from the command line for one off prompts and also have an interactive chat mode, plus a simple HTTP server with an API that feels compatible with OpenAI and Anthropic style requests.

It should be able to pull models from Hugging Face, let me choose CPU or GPU when available, and support common things like different sampling settings, JSON output, and grammar constrained replies. If possible, include image input for vision models too. I also want a few practical extras like progress when loading models, basic stats in chat, and a quiet mode for piping output into other tools.

Use the current docs online if you need to, and make it run well on macOS and Linux.