NVIDIA Open-Sources Full Run:ai Platform

NVIDIA has made its entire Run:ai platform open source under the NVIDIA DSX AI Factory GitHub organization, providing production-grade GPU orchestration for Kubernetes environments. The release includes four decoupled core components: the CNCF Sandbox project KAI Scheduler licensed under Apache 2.0, the KAI Resource Management system, the Workload Map contract, and the GPU Fractioning Operator, which leverages native CUDA 13.4 capabilities to enforce memory bounds. Eight ecosystem partners, including Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster, are directly embedding these tools into their managed cloud and enterprise offerings.

Alongside the core orchestration stack, complementary open-source projects target key operational bottlenecks. Model Streamer eliminates cold-start delay by streaming model weights directly into accelerator memory, integrated upstream in vLLM and SGLang using the --load-format runai_streamer flag, as well as ModelExpress. In Microsoft Azure benchmarks on eight NVIDIA H100 VMs, Model Streamer achieved 3x-6x faster load times and 80 Gbps steady throughput compared to 40 Gbps with default loaders. Enterprise storage tests show WEKA loading weights in 3.5 seconds (40% faster than local NVMe) and VAST Data hitting 1.7 seconds (60% faster than local NVMe). DDN streamed the 65.5 GB Qwen3-32B model across 8 GPUs in under four seconds, yielding up to 10% in daily GPU runtime savings.

For engineering teams and cloud operators, this open release transforms complex hardware management into flexible, reusable infrastructure code. Meanwhile, the companion project Grove automates multi-component inference architectures and supports hardware like the NVIDIA GB200 NVL72 by managing ComputeDomain resources for multinode NVLink setups. Operators can now adopt the full stack via Helm charts on NVIDIA NGC or integrate isolated tools to enforce hard GPU memory limits, automate tenant boundaries, and streamline multi-tenant cluster management.