thu-ml/tianshou ? reverse-engineered prompt

Build me a Python reinforcement learning library in PyTorch that makes it easy to train agents on Gymnasium environments. I want both a simple high level way to run training on my own environment and a more flexible lower level API for people who want to build or tweak algorithms themselves.

Please include support for common RL methods like DQN, PPO, A2C, SAC, TD3, DDPG, and offline methods like BCQ and CQL, plus replay buffers, n step returns, prioritized replay, and basic imitation learning. It should work with vectorized environments, handle recurrent policies too, and support both single agent and multi agent use cases if possible.

Make it feel clean, well organized, and easy to extend, with good docs and example scripts for training on classic control and MuJoCo style tasks. If you need to, look up the current Gymnasium and PyTorch docs online so the setup and APIs are modern.