eugr/spark-vllm-docker ? reverse-engineered prompt

Reverse engineered prompt

Build me a Docker based setup for running vLLM on DGX Spark, both on a single machine and across a small cluster.

I want one setup that can build the image, copy it to the other nodes, download a model once, and then launch vLLM with the right networking and distributed settings. It should be easy to run in solo mode too, so I can just start serving a model on one Spark without extra steps. Please include simple scripts for building, launching, and downloading models, and make sure the cluster flow works with passwordless SSH and direct node discovery.

It should support fast model loading, common inference flags, and the ability to choose between Ray and vLLM’s distributed mode. If needed, look up current docs online so the launch commands and container settings are up to date.

Are you gonna build this?

make sure you review the code using coderabbit

Try freeSponsored — opens CodeRabbit in a new tab