Kubernetes

2026-11-02

COMPANIES · 1 SOURCE · NVIDIA Developer

NVIDIA launches open-source "NVIDIA Recovery" to automate distributed training recovery

NVIDIA announced NVIDIA Recovery, an open-source workload-level service in the DSX OS portfolio that detects training interruptions via application logs, GPU metrics, and scheduler signals and runs a four-stage loop to attribute failures, isolate nodes, and restart or stop jobs. The service integrates with Slurm via a Helm-deployable Advisor Controller, starts in monitor mode by default, provides an audit trail and Recovery Console, and the GitHub repo and quickstart allow local deployment without physical GPUs; production monitoring reported ~8,550 anomaly detections over 3 months with mean detection and recovery-action latencies of 69s and 94s respectively.

7.0

2026-10-30

COMPANIES · 1 SOURCE · InfoQ AI, ML & Data Engineering

Netflix builds GPU reliability platform with layered health checks and automated remediation

Netflix has built a Kubernetes-based GPU compute platform that uses layered device and network diagnostics (e.g., NVIDIA DCGM, Elastic Fabric Adapter checks), node agents that surface conditions to controllers, progressive automated remediation integrated with Karpenter, and fault-tolerant training (checkpointing, elastic workers). The company reports >99.95% GPU-node availability and job success rising from ~98% to >99.5%, and is also using NVIDIA MIG and preparing for Kubernetes DRA to improve utilization and topology-aware placement.

7.0

2026-10-26

COMPANIES · 1 SOURCE · NVIDIA Developer

NVIDIA Mission Control 2.5 adds autonomous recovery for Kubernetes workloads via Run:ai

NVIDIA Mission Control 2.5 extends its autonomous recovery engine — which detects, diagnoses, and recovers from failures — to Kubernetes-based AI workloads managed by NVIDIA Run:ai. The release introduces a three-layer control loop coordinating the Kubernetes scheduler, job recovery, and hardware recovery services, and notes additional features such as AI-enabled network anomaly detection with documentation and release notes available for upgrading.

6.0

2026-10-22

COMPANIES · 1 SOURCE · The Decoder

Google unveils Agent Compute Engine to run each Gemini agent in hardware‑isolated microVMs

Google announced Agent Compute Engine, a service that runs each AI agent inside its own hardware‑isolated microVM secured by Google's Titanium chip and says it already powers Gemini. The service — which underpins the open‑sourced Agent Substrate Kubernetes runtime — pauses idle VMs to free resources, uses shared memory to cut per‑agent costs, and (Google says) can fit up to ten times as many agents per server; it will open for preview in coming months and adds token tracking, model routing, and third‑party identity provider support extending to Cloud Run, GKE and managed endpoints.

8.0

2026-10-20

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

Apple engineers present scaling framework-agnostic AI agent infrastructure with Ray at QCon AI

At QCon AI, Apple engineers Deepak Mohanakumar Chandramouli and Bhumik Vinodkumar Thakkar described moving AI agents from local FastAPI prototypes to resilient production deployments on Kubernetes using Ray, covering RayCluster topology, stateful memory handling, autoscaling for dynamic traffic, control‑plane designs, multi‑region setups, safety guardrails, and an LLM‑as‑a‑judge evaluation pipeline. They illustrated these operational patterns with a trip‑planner agent example and shared production learnings and observability considerations.

4.0

2026-10-16

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

Talk: 'Exploding GPUs' — AI security is still Kubernetes security (QCon London)

Andrew Martin, founder and CEO of ControlPlane, presented 'Exploding GPUs' at QCon London arguing that securing cloud‑native AI workloads remains fundamentally a Kubernetes problem. He outlined an evolving threat landscape — non‑deterministic agent behavior, GPU supply‑chain and shared‑compute risks, and model escape — and proposed practical defenses such as prompt firewalling and agentic identity for production environments.

5.0

2026-10-12

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

Spotify more than doubled monthly merged code changes as AI-assisted development expanded

Spotify reported monthly merged changes rose from about 8,100 to 17,000 year‑over‑year in August after expanding AI‑assisted development. The company added a background coding agent called Honk (running Claude via the Agent SDK in Kubernetes), expanded fleet automation and Fleetshift orchestration, and tightened monitoring, testing, rollout safeguards and rollback capacity after incidents where higher change volume strained verification controls.

7.0

2026-10-07

RESEARCH · 1 SOURCE · Microsoft Research

Microsoft Research Asia open-sources Agent Lightning v1.0, a lightweight harnessed agentic RL framework

Microsoft Research Asia released Agent Lightning v1.0, a roughly 3,500-line open-source implementation of a Harnessed Agentic RL training paradigm that trains on the same agent harness used in deployment via an LLM proxy. The framework runs agents as native Kubernetes jobs, avoids reimplementing agent harnesses, and includes an end-to-end coding-agent example that raised Qwen3.5-9B’s Pass@1 on SWE-bench Verified from 41.8% to 56.4% using about 6,000 training samples.

7.0

2026-10-06

CODING · 1 SOURCE · NVIDIA Developer

NVIDIA releases AICR v1.0 to standardize GPU Kubernetes cluster configs

NVIDIA published AI Cluster Runtime (AICR) v1.0, a version-locked system that pins validated component combinations and establishes a stable compatibility contract across the CLI, REST API, Go SDK, bundle layout, and artifact schemas for GPU-accelerated Kubernetes clusters. The release includes four capabilities (Snapshot, Recipe, Bundle, Validation), a live validation dashboard with signed evidence, and ecosystem integrations from Pulumi Labs and Mirantis’s k0rdent to consume recipes via different IaC and multi-cluster tools.

7.0

2026-09-23

CODING · 1 SOURCE · NVIDIA Developer

NVIDIA open-sources NVCRE, a Kubernetes controller to validate GPU cluster readiness

NVIDIA Cluster Readiness Engine (NVCRE) is an open-source Kubernetes controller that runs real distributed workloads across topology-aware node groups to validate GPU cluster readiness before production AI jobs start. It exposes Certification/Workflow/Job CRDs, adaptive fault isolation to pinpoint failing nodes, a WorkloadRun API for multi-node GPU setup, integrates with NVIDIA AI Cluster Runtime and NVSentinel (part of NVIDIA DSX OS), and is installable via nvcrectl; contributors can extend tests and workload adapters on GitHub.

7.0

2026-09-23

COMPANIES · 1 SOURCE · NVIDIA Developer

NVIDIA open-sources NodeWright, a Kubernetes-native host package manager for GPU fleets

NodeWright is an open-source, Kubernetes-native package manager that declaratively configures and updates host operating systems across GPU clusters while minimizing disruption to running workloads. It integrates with NVIDIA DSX OS components — including NVIDIA AI Cluster Runtime, NVCRE, and NVSentinel — supports progressive rollout policies, and is installable via Helm.

6.0

2026-09-23

RESEARCH · 1 SOURCE · Microsoft Research

Study: Offloading inference boosts mobile manipulation robots' performance, battery life, and model scale

A primary technical report challenges the default of running inference solely on onboard GPUs for physical AI, showing that offloading inference to edge or cloud GPUs can raise task success rates, enable larger models, and extend battery life across mobile manipulation workloads. The authors quantify large effects—e.g., mapping/planning slowdowns up to 383% on smaller GPUs vs. an A100, navigation detection drops around 30%, accuracy losses up to ~50% for some models, and onboard GPUs increasing battery drain (example: Jetson Thor up to ~160%)—and introduce a Kubernetes-based Physical AI Toolchain for containerizing and orchestrating distributed robotics inference.

7.0

2026-09-23

STARTUPS · 1 SOURCE · InfoQ AI, ML & Data Engineering

Modal rebuilds sandbox infrastructure to scale 1 million concurrent sandboxes in seconds

Modal engineers rebuilt their sandbox platform to avoid centralized coordination (e.g., etcd-backed Kubernetes control planes), making each worker the source of truth and running many parallel schedulers. In benchmarks the new architecture created 1 million sandboxes in under a minute with median startup-to-code under 0.5s and sustained tens of thousands of sandbox creations per second.

7.0

2026-09-22

CODING · 1 SOURCE · NVIDIA Developer

NVIDIA announces Topograph toolkit to publish cluster topology for topology-aware scheduling

NVIDIA Topograph is an open-source toolkit that discovers and normalizes cluster topology from cloud APIs and on‑prem fabrics, then publishes that model as Kubernetes node labels, Slurm configuration, or Slinky ConfigMaps so workload managers can perform topology-aware scheduling. It supports several cloud providers and on‑prem fabrics (InfiniBand, Spectrum‑X, Multi‑Node NVLink), integrates with DSX OS components such as DRA and KAI Scheduler, and is available from the dsx-ai-factory/topograph GitHub repo with Helm and native Slurm deployment options and simulation utilities.

6.0

2026-09-22

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

Google open-sources AX, a Kubernetes-style orchestrator for autonomous AI agents

Google released AX (google/ax), an Apache‑2.0‑licensed, open-source orchestrator and declarative runtime for autonomous AI agents built atop Agent Substrate. AX treats agents as stateful actors with sub-second suspend/resume, exposes four Kubernetes-style CRD primitives (Task, Workspace, Gateway, Model) via ax.io/v1alpha1, and provides an ax CLI for managing sandboxed agent workloads aimed at production-scale and research use cases.

7.0

2026-09-18

COMPANIES · 1 SOURCE · InfoQ AI, ML & Data Engineering

WSO2 launches Agent Manager (GA) to govern multi-framework AI agents

WSO2 announced the general availability of the open-source WSO2 Agent Manager, a centralized platform for identity, governance, security controls, and operational oversight of AI agents across models, frameworks, and deployment environments. The GA release adds deeper agent identity features, governance for Model Context Protocol (MCP) interactions, a Kubernetes-native sandboxed runtime, OpenTelemetry tracing, lifecycle controls (including suspension), and over 40 built-in policies for things like PII masking and rate limiting.

7.0

2026-09-15

CODING · 1 SOURCE · NVIDIA Developer

Scaling federated learning across Docker, Kubernetes, and Slurm with NVIDIA FLARE

An NVIDIA Developer article describes how NVIDIA FLARE can be used to run and scale federated learning workloads across common orchestration environments — Docker, Kubernetes, and Slurm — offering guidance for moving beyond single-server, few-client setups to larger deployments.

5.0

2026-09-09

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

Fixing the AI infra scale problem by stuffing 1M sandboxes in a single server (Unikraft presentation)

Felipe Huici presents how Unikraft uses isolation primitives, Linux kernel optimizations, and snapshotting techniques to enable millisecond cold boots, stateful scale-to-zero, and extreme density for sandboxing AI workloads. He reports being able to maintain sub-10ms performance at scale and describes integration with Kubernetes and hardware-level security.

6.0

2026-09-03

CODING · 1 SOURCE · NVIDIA Developer

How to Carry User Identity Across Federated Kubernetes and AI Platforms

An article on NVIDIA Developer outlines the challenges and approaches for carrying user identity and access information as users move between a central portal, governed datasets, launched notebooks, and other components of federated Kubernetes-based AI platforms. It addresses how identity continuity supports governance and consistent access across distributed services.

6.0