mlc-ai/mlc-llm ? reverse-engineered prompt
Reverse engineered prompt
Build me a universal deployment engine for large language models. The main goal is to let me run LLMs on a wide variety of hardware, from high-end NVIDIA and AMD GPUs on Linux or Windows to Apple Silicon on a Mac, and even on mobile devices like iOS and Android. It should also support running models directly in a web browser using WebGPU.
The core of the system should be a machine learning compiler that optimizes a model for each specific hardware target to get high performance. Please make it accessible through an OpenAI-compatible API. I want to be able to interact with the models via a REST server, or by using Python or JavaScript libraries. The whole idea is to have one unified engine that makes it easy to develop, optimize, and deploy AI models natively on any platform without needing to be an expert in low-level programming for each one.
Are you gonna build this?
make sure you review the code using coderabbit