GPU scheduling in Kubernetes used to be deceptively simple: install the NVIDIA device plugin, request nvidia.com/gpu: 1, and let the default scheduler find a node with one available GPU. That model got many clusters into production, but it encoded the wrong abstraction. A modern GPU is not just an integer. It has memory size, architecture, interconnect, topology, partitioning modes, sharing modes, and health state.
Related reading: Cluster Autoscaler vs Karpenter.
Dynamic Resource Allocation (DRA) is Kubernetes’ answer to that mismatch. The core DRA APIs graduated to GA in Kubernetes 1.34, with the stable resource.k8s.io/v1 API enabled by default. In 2026, this matters because the ecosystem around AI infrastructure has also moved: NVIDIA donated its DRA Driver for GPUs to the Kubernetes community under CNCF governance at KubeCon Europe 2026, Kueue has native concepts for DRA-aware quota management, KAI Scheduler is a CNCF Sandbox project for large GPU fleets, and inference stacks such as vLLM and KServe are becoming the runtime layer above the scheduler.
This is not a “replace one YAML key with another” migration. DRA changes where device knowledge lives. Instead of asking Kubernetes for a count of opaque extended resources, workloads request a claim against a class of devices, and the scheduler allocates a concrete device that satisfies the claim.
Why the device-plugin model is reaching its limits
The Kubernetes device plugin framework exposes vendor devices to the kubelet. For NVIDIA GPUs, the traditional resource name is nvidia.com/gpu, requested in container resources.requests and resources.limits. This remains useful, especially for simple clusters.
The limitation is in the API shape. Kubernetes extended resources are integer resources and cannot be overcommitted. The Kubernetes documentation also states that devices cannot be shared between containers through the basic extended-resource model. That is fine for “one pod owns one whole GPU”. It is much weaker for LLM inference, mixed training queues, fractional capacity, MIG profiles, topology-sensitive multi-GPU jobs, and heterogeneous node pools.
The device-plugin model also pushes too much meaning into out-of-band policy. If you need A100s rather than L4s, you usually add node labels, node affinity, taints, or separate node groups. If you need a MIG slice, you configure GPU Operator and device plugin strategy, then expose separate resource names or labels. If you need low-latency multi-GPU placement, you combine scheduler plugins, topology labels, and workload-specific conventions.
Those workarounds fragment the source of truth. The scheduler sees integer capacity. The driver knows device details. The autoscaler knows node templates. The ML platform knows model requirements. DRA gives Kubernetes a structured device model so those systems can coordinate through API objects.
For node provisioning, this does not remove the need for good autoscaling. You still need a node autoscaler that can bring up GPU capacity with the right labels, taints, AMI, driver stack, and instance family. If you run on AWS, /cluster-autoscaler-vs-karpenter/ is directly relevant: GPU pods sitting Pending are often a node provisioning problem before they are a scheduler problem. On EKS, /eks-auto-mode/ is also worth reading because managed node lifecycle and accelerator support change how much of the stack you own.
What DRA actually adds
DRA introduces Kubernetes APIs for claiming devices. The stable API group is resource.k8s.io/v1. The important objects are:
| API object | Scope | Purpose |
|---|---|---|
DeviceClass | cluster | Defines a category of devices and optional selectors/configuration. Claims reference a DeviceClass. |
ResourceSlice | cluster | Published by DRA drivers. Describes available devices, attributes, capacity, and node access. |
ResourceClaim | namespace | Requests access to devices. The scheduler allocates concrete devices into the claim status. |
ResourceClaimTemplate | namespace | Template for per-pod ResourceClaims, similar in spirit to volume claim templates. |
Pod spec.resourceClaims | pod | Makes a ResourceClaim or ResourceClaimTemplate available to the pod. |
Container resources.claims | container | Attaches a named claim to a specific container. |
The control flow is simple. A DRA driver publishes devices as ResourceSlice objects. A cluster administrator or driver provides DeviceClass objects. A workload creates a ResourceClaim or references a ResourceClaimTemplate. During scheduling, Kubernetes evaluates the claim against ResourceSlices, picks devices on nodes where the pod can run, stores the result in ResourceClaim status, and the driver prepares the device for the pod.
This is close to the PersistentVolumeClaim mental model: a pod references a claim with requirements, and Kubernetes binds it to something concrete. With DRA, the pod claims a device from a class, with selectors and constraints that drivers and the scheduler can reason about.
Do not overstate this: DRA is not live GPU hot-plugging, and it is not a model-serving platform. It is a scheduling-time allocation framework. You still need CPU and memory requests to be correct, as covered in /2026-05-kubernetes-resource-requests-limits/. You still need queueing for batch fairness and an inference runtime for serving.
A minimal DRA GPU request
The exact DeviceClass names and available attributes depend on the installed driver. NVIDIA’s NIM Operator documentation says the NVIDIA DRA driver deploys a default gpu.nvidia.com DeviceClass for physical GPUs. Always verify this in your cluster:
kubectl get deviceclasses
kubectl get resourceslicesFor a single GPU per pod, use a ResourceClaimTemplate:
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: one-nvidia-gpu
namespace: ai
spec:
spec:
devices:
requests:
- name: gpu
exactly:
deviceClassName: gpu.nvidia.com
allocationMode: ExactCount
count: 1
---
apiVersion: batch/v1
kind: Job
metadata:
name: dra-gpu-smoke-test
namespace: ai
spec:
completions: 1
parallelism: 1
template:
spec:
restartPolicy: Never
resourceClaims:
- name: gpu
resourceClaimTemplateName: one-nvidia-gpu
containers:
- name: cuda
image: nvcr.io/nvidia/cuda:12.5.1-base-ubuntu22.04
command: ["nvidia-smi"]
resources:
claims:
- name: gpu
requests:
cpu: "1"
memory: 1Gi
limits:
memory: 1GiThis manifest uses the stable resource.k8s.io/v1 API and the pod-level resourceClaims plus container-level resources.claims fields shown in the Kubernetes DRA task documentation. It deliberately does not use the legacy nvidia.com/gpu resource request.
In production, you will usually add selectors. Kubernetes supports CEL selectors in DRA claims, but attribute names are driver-specific. Inspect ResourceSlices before standardizing selectors:
kubectl get resourceslices -o yamlFor example, a platform team might publish DeviceClasses such as “inference-l4”, “training-h100”, or “mig-1g-10gb” instead of asking application teams to write low-level CEL expressions.
MIG, sharing, and topology awareness
MIG is where DRA becomes more than nicer syntax. With the device-plugin model, MIG support works, but the cluster often ends up with a mixture of resource names, node labels, and operational conventions. DRA lets the driver publish device shapes and capacities in ResourceSlices.
NVIDIA’s current DRA driver documentation is cautious: the driver manages GPUs and ComputeDomains; ComputeDomains are officially supported for robust and secure Multi-Node NVLink, while some GPU allocation features are still described as exploratory in the upstream README. NVIDIA’s NIM Operator documentation already covers Kubernetes v1.34 or later, resource.k8s.io/v1, the gpu.nvidia.com DeviceClass, full GPU versus MIG decisions, and the GPU Operator path. Pin driver versions and test the exact mode you intend to offer.
Topology matters at two levels.
First, there is node-local topology: NUMA locality, PCIe lanes, NVLink, and whether devices are close to the CPUs and memory the pod uses. Kubernetes has long had a Topology Manager in kubelet, but DRA gives the scheduler more structured information before placement.
Second, there is fleet topology: racks, blocks, zones, and inter-node fabric. Kueue’s Topology Aware Scheduling documentation targets AI/ML workloads where pod-to-pod bandwidth affects runtime and cost. DRA handles device allocation; Kueue decides when a workload should be admitted and how scarce quota should be shared.
Preemption belongs in that same layered view. Kubernetes scheduler preemption and Kueue preemption can make room for higher-priority workloads, but DRA itself is the device allocation substrate. In practice, you combine DRA claims, PriorityClasses, Kueue ClusterQueues, and possibly KAI Scheduler policies to get predictable multi-tenant behavior.
What NVIDIA’s CNCF donation changes
On March 24, 2026, NVIDIA announced at KubeCon Europe in Amsterdam that it was donating the NVIDIA DRA Driver for GPUs to CNCF, moving it from vendor governance to community ownership under the Kubernetes project.
It changes the risk profile for platform teams. A GPU DRA driver under Kubernetes community governance is easier to treat as part of the cloud-native substrate, and it creates a clearer collaboration point for cloud providers, Kubernetes SIGs, Kueue, KAI Scheduler, and inference platforms.
It does not make NVIDIA-specific hardware vendor-neutral. CUDA, MIG, NVLink, GPU Operator, and driver lifecycle remain NVIDIA concerns. What improves is the Kubernetes integration surface for advertising, claiming, allocating, and preparing devices.
NVIDIA also announced that KAI Scheduler had been onboarded as a CNCF Sandbox project. CNCF describes KAI Scheduler as a Kubernetes scheduler for optimizing GPU resource allocation for AI workloads in large-scale clusters. DRA models and allocates devices; KAI provides AI-focused scheduling policy; Kueue provides quota, admission, fair sharing, and preemption; KServe and vLLM provide the inference serving layer.
For production inference, vLLM and KServe are consumers of GPU scheduling, not replacements for it. vLLM documents KServe integration for distributed model serving, and KServe documents multi-node, multi-GPU inference using a vLLM serving runtime.
Migration: device plugin to DRA
Do not migrate the whole fleet in one step. A practical migration looks like this:
- Inventory existing GPU workloads. Classify them by device shape: whole GPU training, small inference, MIG-friendly inference, multi-GPU single-node, and multi-node training.
- Upgrade the control plane and nodes to Kubernetes 1.34 or later. Verify
resource.k8s.io/v1is available withkubectl api-resources | grep resource.k8s.io. - Install or upgrade the GPU Operator and NVIDIA DRA driver in a small, isolated GPU node pool.
- Verify
DeviceClassandResourceSliceobjects before writing application selectors. - Define platform-owned DeviceClasses for common use cases. Prefer “h100-training”, “l4-inference”, or “mig-small” over asking every team to understand device internals.
- Convert one non-critical workload from
nvidia.com/gputo aResourceClaimTemplate. Keep CPU and memory requests unchanged unless you are intentionally resizing the workload. - Add Kueue for batch admission if multiple teams compete for GPUs. Use ClusterQueues, ResourceFlavors, cohorts, fair sharing, and preemption policies.
- Validate node provisioning. If the claim is valid but no node exists, the autoscaler still has to create the right GPU node.
- Roll into serving platforms. For KServe/vLLM, verify how the serving controller passes or creates DRA claims.
- Retire legacy paths only after observability, rollback, and quota policies are in place.
During migration, it is reasonable to run legacy device-plugin workloads and DRA workloads side by side on separate node pools. Avoid advertising the same physical GPU through two allocation systems to the same scheduling domain unless the driver documentation explicitly supports that configuration.
Device plugin vs DRA
| Capability | Device plugin / extended resource | DRA |
|---|---|---|
| Workload request | nvidia.com/gpu: 1 integer resource | ResourceClaim or ResourceClaimTemplate |
| API maturity | Device plugin framework exists since Kubernetes 1.10 beta | Core DRA APIs GA in Kubernetes 1.34 |
| Device attributes | Mostly external labels and conventions | Structured attributes/capacity through ResourceSlices |
| Sharing | Limited by extended-resource model; vendor-specific sharing modes | Claims can model sharing patterns when supported by driver and feature set |
| MIG / partitions | Works through vendor configuration and resource exposure | Better fit for requesting specific device shapes |
| Scheduler awareness | Counts resources on nodes | Allocates concrete devices during scheduling |
| Autoscaler visibility | Often depends on node templates and resource names | Structured parameters improve simulation potential, but autoscaler support still matters |
| Best fit | Simple whole-GPU workloads | Heterogeneous, partitioned, shared, or topology-sensitive GPU fleets |
When you do not need DRA yet
DRA is not mandatory for every GPU cluster in 2026.
If every workload needs exactly one whole GPU on a homogeneous node pool, the device plugin model may be simpler. If your managed Kubernetes provider does not support the driver path you need, waiting is rational. If your bottleneck is cold node provisioning, image pull time, model download time, or bad CPU/memory requests, DRA will not fix that by itself.
Pilot DRA when multiple teams compete for expensive GPUs, you run mixed GPU models, MIG or sharing is a first-class requirement, multi-GPU topology affects performance, or you need cleaner integration between scheduling, quota, and AI workload platforms.
FAQ
Is DRA GA in Kubernetes?
The core DRA APIs graduated to GA in Kubernetes 1.34, using resource.k8s.io/v1 and enabled by default. Some surrounding features continued to mature later; for example, Kubernetes documentation in 2026 marks certain DRA task flows as v1.35 [stable] and DRA prioritized lists as v1.36 [stable].
Does DRA replace the NVIDIA device plugin?
For DRA workloads, the claim replaces the nvidia.com/gpu extended-resource request. Operationally, many clusters will run both models during migration. The NVIDIA DRA driver is the relevant component for DRA-based GPU allocation.
Can I request fractional GPUs with DRA?
DRA provides a framework for richer requests and sharing, but the exact behavior depends on Kubernetes feature maturity and the driver. NVIDIA documents MIG and time-slicing paths in its GPU stack, while DRA consumable capacity adds a Kubernetes model for sharing device capacity. Validate the exact mode you want before promising fractional GPU self-service.
Is DRA enough for multi-tenant GPU scheduling?
No. DRA allocates devices. For fair sharing, quota, admission, and preemption across teams, add Kueue or a scheduler layer such as KAI Scheduler, depending on your workload shape.
Does DRA help with inference?
Yes, but indirectly. DRA makes the GPU request more accurate. vLLM and KServe still handle serving concerns such as model runtime, scaling, routing, and multi-node inference patterns.
CTA: pilot DRA in one GPU node pool
The right first step is one GPU node pool. Install the NVIDIA DRA driver, verify DeviceClass and ResourceSlice objects, and convert one smoke-test Job plus one real low-risk workload to ResourceClaimTemplate. Measure scheduling latency, claim allocation status, GPU utilization, failure modes, and autoscaler behavior. If the pilot is boring, expand it to one tenant queue. If it is noisy, fix the platform contract first.
Sources
- Kubernetes v1.34 DRA GA announcement: https://kubernetes.io/blog/2025/09/01/kubernetes-v1-34-dra-updates/
- Kubernetes Dynamic Resource Allocation concepts: https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/
- Kubernetes DRA workload task and YAML fields: https://kubernetes.io/docs/tasks/configure-pod-container/assign-resources/allocate-devices-dra/
- Kubernetes ResourceClaim API reference: https://kubernetes.io/docs/reference/kubernetes-api/resource/resource-claim-v1/
- Kubernetes ResourceSlice API reference: https://kubernetes.io/docs/reference/kubernetes-api/resource/resource-slice-v1/
- Kubernetes device plugin documentation: https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/
- Kubernetes DRA consumable capacity: https://kubernetes.io/blog/2025/09/18/kubernetes-v1-34-dra-consumable-capacity/
- KEP-4381 DRA structured parameters: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4381-dra-structured-parameters
- NVIDIA DRA Driver for GPUs repository: https://github.com/kubernetes-sigs/dra-driver-nvidia-gpu
- NVIDIA KubeCon Europe 2026 DRA driver donation announcement: https://blogs.nvidia.com/blog/nvidia-at-kubecon-2026/
- NVIDIA NIM Operator DRA support: https://docs.nvidia.com/nim-operator/latest/dra.html
- NVIDIA GPU Operator sharing documentation: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html
- CNCF KAI Scheduler project page: https://www.cncf.io/projects/kai-scheduler/
- Kueue overview and DRA concepts: https://kueue.sigs.k8s.io/docs/overview/
- Kueue Topology Aware Scheduling: https://kueue.sigs.k8s.io/docs/concepts/topology_aware_scheduling/
- vLLM KServe integration: https://docs.vllm.ai/en/stable/deployment/integrations/kserve/
- KServe multi-node/multi-GPU vLLM inference: https://kserve.github.io/website/docs/model-serving/generative-inference/multi-node
