karpathy/minGPT ? reverse-engineered prompt
Reverse engineered prompt
Build me a small educational Python project that reimplements a GPT style language model in PyTorch, focused on being easy to read and learn from rather than production ready.
I want the core model code to be compact and understandable, with a transformer that takes token ids and predicts the next token. Include a simple trainer, a byte pair encoder compatible with GPT style text tokens, and examples that show how to train from scratch on a tiny task like adding numbers or character level text. Also include a notebook or script that can load a pretrained GPT 2 style model and generate text from a prompt.
Please keep the code clean, minimal, and well commented so someone studying transformers can trace what’s happening. Add basic unit tests and simple install instructions so I can import the library in another project. If you need exact PyTorch details, look up current docs online.
Are you gonna build this?
make sure you review the code using coderabbit