hyperionaidev/smolmaxxing ? reverse-engineered prompt
Reverse engineered prompt
Build me a small project that can run surprisingly big language models on very limited devices, especially old Android TV boxes, an old iPad, and Android phones with not much free memory.
I want it to feel like a real working demo, not just a benchmark. It should let me load a prepared model, chat with it locally, and show the token stream live in a simple web page or app. The important part is that it keeps working even when the model is bigger than RAM, by being smart about memory and reading what it needs from flash as it goes.
Please include the special model files, the scripts or tools to rebuild them, and the patches needed so the runtime can handle the smaller devices. If anything needs current setup details from the upstream docs, look that up online and follow the latest instructions. Also include clear notes on how to run it on each device and how to check the speed.
Are you gonna build this?
make sure you review the code using coderabbit