ggml-org/ggml ? reverse-engineered prompt

Reverse engineered prompt

I want to build a high-performance C++ library for machine learning, something like the core engine that powers projects like llama.cpp. The main goal is to run large language models efficiently on regular consumer hardware, not just big servers. A key feature should be excellent support for integer quantization to make this possible.

It needs to be a low-level, cross-platform tensor library that is really fast and doesn't have any third-party dependencies, keeping it lean and simple to compile. It should also have built-in capabilities for automatic differentiation and some common optimizers like ADAM.

Please set up the project and build the core library. Also, make sure to build the examples so I can see it in action, especially the one for running GPT-2 inference. I'd like to be able to test it with a small model right away.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab