vivekkalyanarangan30/llm_from_scratch ? reverse-engineered prompt

Build me a hands on Python repo that teaches how to build an LLM from scratch in PyTorch, step by step, with small runnable examples for each stage.

I want it organized as a clear curriculum, starting with the basics of a Transformer, then training a tiny language model, then modern upgrades like RMSNorm, RoPE, SwiGLU, KV cache, BPE tokenization, mixed precision, checkpointing, and logging. After that, include simple working examples for mixture of experts, supervised fine tuning, reward modeling, PPO based RLHF, and GRPO. Keep the code easy to read, well commented, and focused on learning how the pieces fit together.

Make sure each part can run on its own, with example scripts, minimal datasets or toy data where needed, and a clean README that explains how to set up the environment and follow the lessons. If you need to check current PyTorch or training best practices online, go ahead and look them up.