gazbg/poormansllama ? reverse-engineered prompt
Reverse engineered prompt
Build me a llama.cpp based inference setup that runs well on a single 8GB or 16GB GPU, with a focus on getting the biggest possible context on older cards like a GTX 1070. I want a simple setup wizard that checks the machine, offers to install anything missing, builds the server, downloads a model, and then creates a ready to run start script with the best flags it found. The server should keep most weights in host RAM, use VRAM smartly for the experts it needs most, and load vision support only when a request actually needs it. It should feel fast for long prompts and normal chat, and it should be practical for agent style work on limited hardware. Make it work on Windows and Linux if possible, and if you need current docs or build details, look them up online.
Are you gonna build this?
make sure you review the code using coderabbit