POLICY · MODELS · #1082
GKE Pod snapshots cut model startup latency up to 89%; GA on 1.35.3-gke.1234000+
Google published benchmarks for GKE Pod snapshots (general availability in May on clusters running 1.35.3-gke.1234000 or later) that checkpoint and restore full pod state including CPU/GPU memory, reporting startup latency reductions up to 89% (a 70B-parameter model in 37s, an 8B model in 15s). The feature relies on gVisor and GKE Sandbox (Autopilot has it by default), stores snapshot data in Cloud Storage, and is configured via PodSnapshotStorageConfig and PodSnapshotPolicy; compatibility rules and application rehydration limitations mean operational work shifts to snapshot lifecycle management.
KEY POINTS
- Google published benchmarks for GKE Pod snapshots (general availability in May on clusters running 1.35.3-gke.1234000 or later) that checkpoint and restore full pod state including CPU/GPU memory, reporting startup latency reductions up to 89% (a 70B-parameter model in 37s, an 8B model in 15s).
- The feature relies on gVisor and GKE Sandbox (Autopilot has it by default), stores snapshot data in Cloud Storage, and is configured via PodSnapshotStorageConfig and PodSnapshotPolicy; compatibility rules and application rehydration limitations mean operational work shifts to snapshot lifecycle management.
- This materially affects ML deployments by drastically lowering cold-start times for large models while creating new operational and compatibility challenges around snapshot lifecycle, storage, and secure rehydration.
WHY IT MATTERS
This materially affects ML deployments by drastically lowering cold-start times for large models while creating new operational and compatibility challenges around snapshot lifecycle, storage, and secure rehydration.