ggml-org/llama.cpp ? reverse-engineered prompt

I want to build a tool in C++ so I can run open-source large language models locally. The main goal is to get really good performance without a ton of setup, and it should work on all kinds of hardware — it needs to be optimized for my Mac with Apple Silicon but also support PCs with NVIDIA or AMD GPUs.

It should be able to handle quantized models to save memory, so I can run bigger models on my machine. I'd like to be able to just download a model from Hugging Face, like a GGUF version of Llama 3 or Mistral, and run it from the command line.

Please also include a server mode that exposes an OpenAI-compatible API, so other apps can talk to it. And if there's a way to include a simple web UI for chatting, that would be perfect. The whole thing should be a self-contained C/C++ project without a lot of external dependencies.