NVIDIA open-sources Run:ai platform under Apache 2.0

NVIDIA has fully open-sourced its Run:ai platform under the Apache 2.0 license on GitHub, incorporating the stack into the NVIDIA DSX AI Factory platform. The platform includes a control plane, cluster services, CLI, installer, and a Model Context Protocol server. NVIDIA divided the architecture into four standalone modules: the CNCF Sandbox KAI Scheduler for batch and gang scheduling, the CUDA 13.4-native GPU Fractioning Operator enforcing strict memory limits, KAI Resource Management for managing quotas and node pools, and the Workload Map Go library supporting PyTorchJob, RayCluster, and KServe types.

For local testing, engineers can spin up the environment without physical accelerators using a single local-dev.sh script. This process builds around 50 Go services inside containers to deploy a 5-node Kind cluster running Kubernetes v1.32.5 alongside 8 simulated Tesla-A10 GPUs. System requirements for Linux hosts specify 4 CPUs, 16 GB of RAM, and 100 GB of free storage, while macOS Podman virtual machines require 12 GB of RAM. Compiling the stack generates roughly 44 GB of cached files. Teams can deploy directly from source or via the NGC Community Product Helm chart without licensing fees, though enterprise support remains tied to paid NVIDIA AI Enterprise tiers.

Alongside the core platform, NVIDIA released Model Streamer and Grove. Model Streamer integrates with vLLM and SGLang using the --load-format runai_streamer flag. In Microsoft testing on 8 NVIDIA H100 VMs hosted on Azure, Model Streamer accelerated weight loading 3x to 6x compared to default methods while doubling throughput from 40 Gbps to 80 Gbps. Grove handles multi-component inference tasks like prefill and decode as all-or-nothing units. Additionally, eight DSX ecosystem partners—Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster—are integrating the open-source release into their offerings.

Unlike lightweight Kubernetes controllers such as Volcano, Kueue, or Apache YuniKorn that focus strictly on job queuing or scheduling, Run:ai offers a full management stack featuring a web interface, REST API, and multi-cluster control plane. For infrastructure practitioners, this shift provides full visibility into workload orchestration code while allowing self-hosted deployments to guarantee cluster splits, such as allocating a 50% guaranteed quota to specific research teams without vendor lock-in.