karpathy/nanochat ? reverse-engineered prompt

Reverse engineered prompt

I want to build a simple setup for training my own small language models, based on Andrej Karpathy's nanochat project. The main idea is to reproduce the "GPT-2 speedrun" that's described, which should let me train a custom chatbot for a relatively low cost on a single powerful GPU node.

Please create a complete Python project that handles the entire pipeline: data tokenization, model pretraining, and then running inference. After the model is finished training, which I know might take a few hours, I want to be able to launch a simple web UI. This web interface should look like a basic version of ChatGPT, where I can type messages and get responses from the model I just trained.

Set up the project with all the dependencies using uv, as the repo suggests. The end goal is to have a system where I can kick off the training process, and once it's done, easily start the chat server to talk to my new AI.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab