JustVugg/colibri — reverse-engineered prompt

Build me a tiny local runtime for GLM 5.2 that can actually run on a normal computer with around 25 GB of RAM by keeping the dense part in memory and streaming the MoE experts from disk as needed. I want the core to be plain C and self contained, with no heavy runtime dependencies, and it should feel honest about speed and memory instead of silently changing model behavior just to fit.

It should let me convert the official weights into a smaller local format, then start a simple chat session from the terminal and ideally also have lightweight desktop and web front ends if that matches the repo. Make the chat usable with proper tokenization, sampling, speculative decoding if it helps, and support grammar constrained output for things like JSON. It should also keep a warm KV cache so a conversation can resume quickly after restart.

Please wire it up end to end so someone can build it, prepare the model, and chat locally on consumer hardware. Look up current docs online if you need to.