Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4
According to NVIDIA Developer. NVIDIA introduces CUDA 13.4 alongside an open-source GPU Fractioning Operator to enforce per-workload memory limits in Kubernetes environments.
- • CUDA 13.4 delivers application-transparent memory boundary enforcement for GPUs.
- • The open-source GPU Fractioning Operator supervises Multi-Process Service on every GPU node.
- • KAI Scheduler v0.18.0 integrates via its NvFractions GPU-sharing mode using pod annotations.
CUDA 13.4 Memory Enforcement
CUDA 13.4 introduces a memory boundary enforcement that is transparent to applications, offering exact memory amounts, guaranteed minimums with burstable maximums, and a choice between time-slicing and sm-sharing compute modes.
GPU Fractioning Operator
A new open-source Kubernetes operator called gpu-fractioning is hosted under the KAI Scheduler organization on GitHub. It runs next to the NVIDIA GPU Operator to set up and run MPS on every GPU node, apply workload memory limits, and measure usage per fraction.
KAI Scheduler Integration
Starting with v0.18.0, KAI Scheduler features a GPU-sharing mode called NvFractions. The scheduler selects which slice of a GPU a workload gets and writes per-container Pod annotations, which the fractioning operator then enforces on the node.
- 1. developer.nvidia.comOpen source