apache/flink ? reverse-engineered prompt

I want to build a powerful data processing engine for big data applications. The system needs to handle both real-time data streams as they come in, and also process large, pre-existing batches of data. It should be really fast, with high throughput and low latency being key.

A major requirement is reliability. It has to be fault-tolerant, meaning it won't lose data or process the same information twice even if a machine goes down. The main way to write jobs for this engine should be using Java, with clean and easy-to-use APIs. Support for Python and Scala would be a big plus. It also needs to connect to other common big data tools like Hadoop, YARN, and message queues like Kafka. Let's make this a flexible, open-source framework for both streaming and batch jobs.