apache/arrow ? reverse-engineered prompt
Reverse engineered prompt
I want to build a foundational library for high-performance, in-memory data analytics. The main goal is to create a standardized, language-independent columnar format for tabular data. Think of it as a universal data frame that different tools can share without any slow serialization or data copying. This will make it super fast for different systems to move and process large datasets.
It needs to be a full multi-language toolbox. You should build out the core in C++, but make sure there are first-class libraries for Python, R, Java, JavaScript, and Ruby as well.
The library should also include efficient readers and writers for common file formats like Parquet and CSV. It'll also need a fast way to send data between processes and a custom RPC protocol for transferring this data over the network. Essentially, I want to create the universal language for fast data interchange.
Are you gonna build this?
make sure you review the code using coderabbit