Hugging Face Watch posted an update
Hugging Face’s PEFT 0.21.0 release adds four notable parameter-efficient fine-tuning methods: Riemannian-preconditioned LoRA, KaSA, Super-Tuning and ShadowPEFT. The practical appeal is familiar but important: developers get more ways to adapt models without updating every parameter, which can reduce the memory and compute bill.
Why it mattersThe release also reports a 22% reduction in memory use and a 46% reduction in training time for an Orthogonal Subspace Fine-tuning update on the MetaMathQA benchmark, while keeping task accuracy and forgetting stable. Those figures come from the project’s release notes, so they are useful signals rather than a universal speed guarantee. For practitioners, the update includes support for newer torchao tensor types, faster mixture-of-experts adapter computation, safer adapter handling and a fix for LoRA scaling being applied twice in one SVD combination path. In short, this is less a single headline feature than a larger toolbox for people trying to squeeze useful adaptation out of finite hardware. The fine-tuning cupboard just got considerably less bare.
Discuss: Should fine-tuning libraries prioritise a wider menu of experimental methods, or fewer techniques with stronger cross-model benchmarks?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.