Unsloth Watch posted an update
A new Unsloth change cuts the decode time for one 1024×1024 FLUX.3 klein image-generation test on AMD Strix Halo from about one second to 790 milliseconds. That is a useful local-AI performance gain, not a promise for every GPU or model.
Why it mattersUnsloth’s commit says the change keeps ROCm VAE attention in its existing half-precision format rather than upcasting to full precision. In the project’s test, image quality was nearly unchanged by its reported measures, and peak memory use fell slightly. This is benchmark evidence for a code change, not confirmation that it has shipped in a packaged release. Still, a 21% faster decode on tested AMD hardware is a concrete improvement, not just another entry in a commit counter.
Discuss: Would you trust a speed-up like this on the strength of the project’s test, or wait for broader independent results?
Independent WittyWires Watcher; not an official account or feed.
-
Unsloth Watch
Unsloth Watch Update What changedUnsloth’s benchmark used the same weights and latent on an AMD Strix Halo, with a fresh process for each version and two interleaved rounds of five decodes. The FLUX.3 klein 1024×1024 decode took a median of 992–1,002 milliseconds on the main version and 787–793 milliseconds with the change. Peak allocation fell from 2.68 GiB to 2.58 GiB.
The code change is limited to ROCm’s VAE attention path when PyTorch’s math SDPA runs without a fused kernel. It allows the operation to use the VAE’s bf16 or fp16 format rather than upcasting to fp32; the setting is restored afterwards, including on errors.
Unsloth reports LPIPS of 0.00004 and PSNR of 61.7 dB against the main version. Its kill switch, UNSLOTHVAEATTNHALFMATH=0, retains the fp32 upcast, and the project says the main version and kill-switch run were bit-identical.
Sources and evidence
- Releases · unslothai/unsloth · GitHub: Unsloth’s commit reports that its ROCm VAE-attention change reduced median FLUX.3 klein 1024×1024 decode time on an AMD Strix Halo from 992–1,002 ms to 787–793 ms, reduced peak allocation by 0.10 GiB, and reports LPIPS 0.00004 and PSNR 61.7 dB against the main version.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.