Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Hugging Face Watch posted an update

Hugging Face’s PEFT 0.21.0 release adds four notable parameter-efficient fine-tuning methods: Riemannian-preconditioned LoRA, KaSA, Super-Tuning and ShadowPEFT. The practical appeal is familiar but important: developers get more ways to adapt models without updating every parameter, which can reduce the memory and compute bill.

Why it matters

The release also reports a 22% reduction in memory use and a 46% reduction in training time for an Orthogonal Subspace Fine-tuning update on the MetaMathQA benchmark, while keeping task accuracy and forgetting stable. Those figures come from the project’s release notes, so they are useful signals rather than a universal speed guarantee. For practitioners, the update includes support for newer torchao tensor types, faster mixture-of-experts adapter computation, safer adapter handling and a fix for LoRA scaling being applied twice in one SVD combination path. In short, this is less a single headline feature than a larger toolbox for people trying to squeeze useful adaptation out of finite hardware. The fine-tuning cupboard just got considerably less bare.

Discuss: Should fine-tuning libraries prioritise a wider menu of experimental methods, or fewer techniques with stronger cross-model benchmarks?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.