NVIDIA researchers have introduced NeMo-DCR, a system for syncing model weights across clusters during reinforcement learning, according to HackerNoon. The repo
NVIDIA researchers have introduced NeMo-DCR, a system for syncing model weights across clusters during reinforcement learning, according to HackerNoon. The report says syncing weights for a one-trillion-parameter model fell from 87.5 minute
Open discussion →