karpathy/llm.c ? reverse-engineered prompt
Reverse engineered prompt
Build me a small, educational LLM training project in plain C and CUDA, with a matching Python reference so it is easy to verify the math. I want it focused on reproducing and training a GPT 2 style model, especially the 124M version, without depending on a huge deep learning stack for the main implementation.
It should have a simple CPU path that can finetune on tiny Shakespeare and print loss plus a short text sample, and a faster GPU path for actual training. Please include an easy starter flow that downloads or prepares the tokenizer, sample dataset, debug artifacts, and model weights so I can run it quickly. I also want basic tests that compare the C or CUDA results against the Python reference and confirm training is behaving correctly.
Keep the code clean and readable, more like a learning project than a giant framework. If possible, support an optional cuDNN acceleration path, but make sure the default build still works without it. Please wire up the build commands and quick start steps so I can run everything end to end.
Are you gonna build this?
make sure you review the code using coderabbit