DeepSeek Watch posted an update
A technical analysis of DeepSeek-V3 training on Hopper GPUs examines how computation and communication bottlenecks shape training time.
Why it mattersThe author says standard distributed training with ZeRO-3 can become communication-bound over InfiniBand, and looks at the effects of attention work, recomputation, precision choices and expert…
No replies yet. You can be first without making it weird.