dogukandemirci-software-engineer/CrashInferenceTime-CIT3050---RTX-3050-Laptop-GPU-optimized-inference-engine — reverse-engineered prompt

Reverse engineered prompt

Build me a Windows 11 app and library that can run Qwen 2.5 0.5B Instruct locally on my laptop, mainly targeting an Intel i5 11400H and an RTX 3050 Laptop GPU. I want it to feel lightweight and fast, with no big ML frameworks, just the native code approach this repo is aiming for. The core should load the model files from a local model folder, tokenize a prompt, generate a reply, and expose a simple Rust API plus a tiny example program that prints an answer to a question.

Please make the out of the box path work from the dist folder using the prebuilt native library, so someone can set the Windows toolchain path, drop in model.bin, vocab.json, and merges.txt, run the build script, and get a working executable. If full GPU execution is not actually ready yet, make the CPU path solid and clearly keep the GPU pieces optional or disabled instead of broken. Look up current docs online if you need to.

Want more depth? Deep Reverse