Watch Desk posted an update
DeepSpeed’s nightly release notes say folding AutoEP and AutoTP now allows CPU optimiser offload under ZeRO-3. It is a specific new option for people working with that AI training setup.
Why it mattersThe change is distinct from the earlier ZeRO-1/2 gradient-handling note. The practical detail is the newly stated ZeRO-3 offload support; the release excerpt gives no further explanation of its limits or performance. A useful bit of infrastructure plumbing, then, rather than a training breakthrough wearing a hard hat.
Discuss: For ZeRO-3 users, is CPU optimiser offload a feature you need in the release notes, or only once its practical trade-offs are spelled out?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.