AWS says lazy-loaded SageMaker images cut AI job starts from minutes to seconds

SageMaker AI can now start training and processing jobs before downloading their full container images, AWS says, turning image setup from minutes of paid waiting into seconds. In AWS’s tests, the change saved one to six minutes per job, with the largest gains on bigger images.

AWS AI Watch analysis

What happened

AWS has introduced Seekable OCI (SOCI) indexing for SageMaker AI Training and Processing jobs. With an indexed image, SageMaker fetches the files needed to start the container first, then streams the rest as the job runs. Teams can use pre-indexed AWS Deep Learning Containers or add an index to their own image. AWS says existing job definitions do not need to change.

In AWS’s measurements, downloading an image compressed to 13.2 GB took about 3 minutes 42 seconds normally, compared with about six seconds using lazy loading. Across eight images from 1.8 GB to 35.4 GB, AWS measured download times of 3 to 11 seconds with SOCI indexing. It says jobs in its tests finished one to six minutes sooner overall. From the second training epoch onwards, performance was unchanged within run-to-run variation; the first epoch took an extra 3 to 16 seconds while it fetched files used for the first time. AWS’s announcement and adoption guide has the measurements and setup details.

Why it matters

The gain is especially useful for short experiments, hyperparameter sweeps and processing jobs, where downloading a large image can consume a noticeable share of the run. Less waiting also means less billed job time, according to AWS. The practical catch is that the benefit depends on what a workload reads at startup: if it immediately touches most of the image, there is less downloading left to do in the background.

Our read

This is an infrastructure improvement with a pleasingly direct payoff: less time watching a progress bar, and potentially less money spent watching it. Teams already using SageMaker should check whether their images and workloads suit lazy loading before changing anything. AWS says jobs fall back to standard downloads when indexing is absent or unsuccessful, so the change is designed not to break the existing route.

What to watch

• How much time SOCI indexing saves on workloads beyond AWS’s published tests.
• Whether teams see meaningful savings on smaller images, where the improvement may be modest.
• Which pre-indexed container images and framework variants AWS makes available.

Discussion spark: Would you adopt lazy-loaded images for a modest per-job saving, or only if your own workload tests showed a clear cost reduction?

not affiliated with or endorsed by Amazon Web Services (AWS)