GUIDE · CODING · #3908
Walkthrough: NVIDIA Run:ai GPU Fractioning enforces per‑Pod GPU‑memory sharing on Amazon EKS (A10G validated)
NVIDIA Run:ai’s open-source release includes a GPU Fractioning Operator that integrates with Kubernetes scheduling to enforce per‑Pod GPU‑memory contracts and export per‑Pod metrics. The walkthrough validates two Pods sharing a single NVIDIA A10G on Amazon EKS (each with an 8 GiB memory contract, successfully allocating 6 GiB), using KAI Scheduler, MPS, and custom AL2023 images; it tests memory limits but not fault containment or compute performance.
KEY POINTS
- NVIDIA Run:ai’s open-source release includes a GPU Fractioning Operator that integrates with Kubernetes scheduling to enforce per‑Pod GPU‑memory contracts and export per‑Pod metrics.
- The walkthrough validates two Pods sharing a single NVIDIA A10G on Amazon EKS (each with an 8 GiB memory contract, successfully allocating 6 GiB), using KAI Scheduler, MPS, and custom AL2023 images; it tests memory limits but not fault containment or compute performance.
- Enforced per‑Pod GPU‑memory sharing on non‑MIG GPUs can increase usable GPU density for notebooks, short evaluations, and small inference services on EKS without hardware partitioning.
WHY IT MATTERS
Enforced per‑Pod GPU‑memory sharing on non‑MIG GPUs can increase usable GPU density for notebooks, short evaluations, and small inference services on EKS without hardware partitioning.