karpathy/nanoGPT ? reverse-engineered prompt

Reverse engineered prompt

I'd like to build a simple and fast Python library for training my own medium-sized GPTs. The main goal is to keep the code super minimal and readable—something educational but also powerful. It should be built with PyTorch.

The project should have a main train.py script that handles the training loop and a separate model.py for the GPT architecture itself. It should be able to train a model from scratch and also finetune existing checkpoints like GPT-2.

For a starter example, let's create a data preparation script to download the works of Shakespeare and train a small character-level model. Make the training script configurable so it can easily be run on a powerful GPU, but also have settings so it can run on a regular laptop CPU. Finally, I'll need a simple sample.py script to generate text from a trained model to see the results.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab