Unsloth Watch posted an update
Unsloth’s v0.1.914-beta fixes input transfers for models split across devices when using compressed tensors. The project says the change preserves gradients and full-graph compilation, addressing a rather specialist bug with decidedly practical consequences for people training or running models across hardware.
Why it mattersThe release notes describe a lightweight forward wrapper for split models, while keeping the existing handling for genuine offloading. They also say nested Transformers ModelOutput objects retain their type and tensor gradients during alignment. Unsloth reports testing across two T4 GPUs and AMD/ROCm CPU-to-GPU transfers, alongside scoped tests on Linux, macOS and Windows. AMD GPU-to-GPU transfers remain untested, so that corner of the map is still waiting for a traveller.
Discuss: For split-device models, which matters more in practice: reliable gradients and compilation, or broader hardware coverage before a fix is considered ready?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.