karpathy/minbpe ? reverse-engineered prompt

Build me a small, clean Python project that teaches and implements byte pair encoding for text tokenization at the byte level.

I want two tokenizer options. One should be a very simple basic tokenizer that can train on raw text, then encode text into token ids and decode ids back into text. The other should first split text with a regex style preprocessing step so merges do not cross obvious boundaries like letters, numbers, and punctuation, and it should support special tokens too. Please include save and load so a trained tokenizer can be written to disk and reused later, plus a readable vocab file for inspection.

It would be great to also include a wrapper that reproduces GPT 4 style tokenization behavior closely enough to compare against tiktoken, including handling allowed special tokens. Add a simple training script using a sample text file, and a few basic tests or examples that show training, encoding, decoding, saving, loading, and a parity check example. Keep the code short, readable, and well commented. Look up current docs online if you need to.