spotify/luigi ? reverse-engineered prompt

Build me a Python library for managing long running batch pipelines, where I can define tasks in Python, set dependencies between them, and have the system figure out what needs to run next.

I want it to handle workflow orchestration, failure handling, a simple command line interface, and a web page that shows the task graph so I can see what is done and what is still waiting. It should work well for local jobs, file based outputs, and also have support for Hadoop style jobs and other common data pipeline tasks.

Please include a few simple example pipelines and make the project easy to install and run from the command line. If you need to look up current docs online for anything unclear, do that. Keep the code clean, tested, and ready to grow into something people could actually use for production batch processing.