NEWS · CODING · #2681
KAI Scheduler v0.18.0 integrates GPU Fractioning Operator to use CUDA 13.4 memory enforcement for fractional GPU sharing
The open-source gpu-fractioning operator (hosted under the KAI Scheduler org) runs alongside the NVIDIA GPU Operator to enforce per-workload GPU memory limits and compute modes using CUDA 13.4's new transparent memory boundary. KAI Scheduler v0.18.0 adds an NvFractions mode that writes per-container annotations (memory size/limits, compute mode, device assignment) which gpu-fractioning enforces on each node; administrators enable it at install time and workloads declare fractional GPU requests via Pod annotations.
KEY POINTS
- The open-source gpu-fractioning operator (hosted under the KAI Scheduler org) runs alongside the NVIDIA GPU Operator to enforce per-workload GPU memory limits and compute modes using CUDA 13.4's new transparent memory boundary.
- KAI Scheduler v0.18.0 adds an NvFractions mode that writes per-container annotations (memory size/limits, compute mode, device assignment) which gpu-fractioning enforces on each node; administrators enable it at install time and workloads declare fractional GPU requests via Pod annotations.
- This stitches CUDA 13.4's node-level memory enforcement into Kubernetes workflows, enabling reliable per-workload GPU memory guarantees and denser packing of AI workloads on shared GPUs.
WHY IT MATTERS
This stitches CUDA 13.4's node-level memory enforcement into Kubernetes workflows, enabling reliable per-workload GPU memory guarantees and denser packing of AI workloads on shared GPUs.