Netflix builds GPU reliability platform with layered health checks and automated remediation
Netflix has built a Kubernetes-based GPU compute platform that uses layered device and network diagnostics (e.g., NVIDIA DCGM, Elastic Fabric Adapter checks), node agents that surface conditions to controllers, progressive automated remediation integrated with Karpenter, and fault-tolerant training (checkpointing, elastic workers). The company reports >99.95% GPU-node availability and job success rising from ~98% to >99.5%, and is also using NVIDIA MIG and preparing for Kubernetes DRA to improve utilization and topology-aware placement.