NEWS · COMPANIES · #2672
Amazon SageMaker adds SOCI-indexed images to speed AI training and processing jobs
Amazon announced support for Seekable OCI (SOCI) indexing in SageMaker AI Training and AI Processing jobs. SOCI enables lazy-loading of container images stored in Amazon ECR (you can use pre-indexed AWS Deep Learning Containers or add an index to your own image); SageMaker fetches only files needed to start the container and streams the rest, producing seconds-long startups in AWS benchmarks (measured pull speedups of about 8x–46x and ~34x on a 13.2 GB image across instance types) and up to a reported 90% reduction in GPU idle time, with no changes required to job definitions and automatic fallback to standard pulls.
KEY POINTS
- Amazon announced support for Seekable OCI (SOCI) indexing in SageMaker AI Training and AI Processing jobs.
- SOCI enables lazy-loading of container images stored in Amazon ECR (you can use pre-indexed AWS Deep Learning Containers or add an index to your own image); SageMaker fetches only files needed to start the container and streams the rest, producing seconds-long startups in AWS benchmarks (measured pull speedups of about 8x–46x and ~34x on a 13.2 GB image across instance types) and up to a reported 90% reduction in GPU idle time, with no changes required to job definitions and automatic fallback to standard pulls.
- Reduces billable GPU/VM idle time and job startup latency, lowering costs and speeding iteration for tuning sweeps and short experiments where image pulls previously dominated runtime.
WHY IT MATTERS
Reduces billable GPU/VM idle time and job startup latency, lowering costs and speeding iteration for tuning sweeps and short experiments where image pulls previously dominated runtime.