eleutherai/gpt-neox ? reverse-engineered prompt

Build me a project for training and running large language models on GPUs, with a simple way to start training from a config file, generate text from a checkpoint, and evaluate the model on common language tasks.

I want it to support distributed training across multiple GPUs and multiple machines, and be usable on big cluster setups too. It should include ready to use configs for popular model styles, plus the ability to plug in custom datasets, tokenizer setups, and experiment tracking like Weights and Biases or TensorBoard. Please make sure there are clear scripts for training, generation, evaluation, and data prep, and that the setup works well with modern GPU training tools and mixed precision.

If anything needs current setup details from external docs, look them up online and follow the latest guidance.