NVIDIA Open-Sources Run:ai Under Apache 2.0 for Kubernetes GPU Orchestration

The release makes Run:ai part of the NVIDIA DSX AI Factory platform, and platform teams can now inspect, fork and self-host the stack, according to the report. The monorepo includes the control plane, cluster services, installer, command-line interface and an MCP server for AI assistants. A local development build compiles about 50 Go services. The platform runs on Kubernetes or OpenShift clusters with NVIDIA GPUs. Local development runs on macOS or Linux with eight simulated Tesla-A10 GPUs and does not require GPU hardware. NVIDIA published no platform-level benchmark for this release. The report said the related Model Streamer loaded models three to six times faster on eight NVIDIA H100 virtual machines on Azure. The community edition has no NVIDIA enterprise support. Local builds on Linux need 16 GB of RAM and 100 GB of disk.

Run:ai is an AI workload and GPU orchestration platform for Kubernetes. It schedules development, training and inference workloads across shared GPU clusters, runs on top of Kubernetes and does not replace it. The control plane handles workload submission, resource management and monitoring. Cluster services run on each Kubernetes cluster and add GPU-aware scheduling. Users reach the platform through a web UI, CLI, REST API and an MCP server for AI assistants.

The release is split into four components that teams can adopt independently. KAI Scheduler is the Kubernetes-native scheduling engine; it is Apache 2.0 and a CNCF Sandbox project, and it covers gang scheduling, time-based fair-share, topology awareness, bin-packing and DRA support. GPU Fractioning Operator brings CUDA 13.4-native GPU fractions to Kubernetes, with the CUDA driver enforcing a hard memory limit per workload. NVIDIA retired its older proprietary fractioning in favor of this approach; the operator is scheduler-agnostic and exports per-pod metrics. KAI Resource Management adds projects, departments, quotas and node pools, and NVIDIA’s example guarantees 50% of a cluster to a research team. Workload Map is a Go library that describes workload types such as PyTorchJob, RayCluster and KServe services; it works offline and does not schedule, queue or autoscale.

The local development guide uses one script, local-dev.sh. It builds about 50 Go services from source inside containers and creates a five-node Kind cluster pinned to Kubernetes v1.32.5. A fake GPU operator simulates eight Tesla-A10 GPUs. Supported hosts are macOS or Linux on amd64 or arm64; Windows is not supported. Linux hosts need four CPUs, 16 GB of RAM and 100 GB of free disk, while macOS Podman virtual machines need at least 12 GB of RAM. A full build leaves about 44 GB of caches. The --gpu-operator flag switches to real NVIDIA GPUs; that mode needs Linux, Docker and the NVIDIA Container Toolkit. AMD and Intel accelerators are not supported.

Teams can deploy in production from source or from the NGC Community Product Helm chart. That edition needs no paid subscription but excludes NVIDIA enterprise support. Teams can also adopt it through eight DSX ecosystem partners: Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud and vCluster. Supported commercial editions continue through NVIDIA AI Enterprise, NVIDIA Mission Control and select cloud marketplaces.

Two adjacent open-source projects round out the stack. Model Streamer streams model weights directly into GPU memory; vLLM and SGLang enable it with --load-format runai_streamer. Microsoft’s published benchmark used eight NVIDIA H100 virtual machines and showed three to six times faster loading than the default loader, with throughput held at 80 Gbps versus about 40 Gbps on the default path. Grove orchestrates multicomponent inference workloads, starting components such as prefill and decode as all-or-nothing units.

MarkTechPost compared Run:ai with Volcano, Kueue and Apache YuniKorn, all under Apache 2.0 licenses. Run:ai is governed under the NVIDIA DSX AI Factory organization, while KAI Scheduler is a CNCF Sandbox project; Volcano is CNCF Incubating, Kueue is under kubernetes-sigs wg-batch, and YuniKorn is under the Apache Software Foundation. Run:ai covers a full platform with control plane, UI, CLI, REST API and MCP server; Volcano is a batch scheduling system, Kueue a job queueing controller, and YuniKorn a scheduler and alternative to the default. The comparison said Run:ai extends kube-scheduler with KAI Scheduler rather than replacing it. It listed gang scheduling for Run:ai, Volcano and YuniKorn, and timeout-based all-or-nothing behavior for Kueue. It listed quota hierarchies of projects, departments and node pools for Run:ai; multi-level queues for Volcano; cohorts and fair sharing for Kueue; and hierarchical queues for YuniKorn. For GPU sharing it listed CUDA 13.4-native fractions with hard memory limits for Run:ai, vGPU and dynamic MIG for Volcano, DRA-based GPU quota with fractioning not in the overview for Kueue, and no disclosure for YuniKorn. It listed topology-aware scheduling for Run:ai, Volcano and Kueue, with no disclosure for YuniKorn.

Editor's Summary

NVIDIA has open-sourced its full Run:ai GPU orchestration platform under Apache 2.0, giving teams self-hosted Kubernetes scheduling, quota management and local development tools. The release includes KAI Scheduler, GPU fractioning and resource management components, while commercial support remains routed through NVIDIA AI Enterprise and partners. The move lowers adoption barriers for platform teams and keeps enterprise support as a paid distinction.