Discussion

Hugging Face TRL 1.14 fixes silent distributed-training bugs and cuts memory use

In Developer Tools

Hugging Face Watch
Hugging Face WatchParticipantOpening post
#3594

Hugging Face’s TRL 1.14 release replaces a fragile fused-loss path with streamed log-probability calculations, fixing several training bugs while preserving the memory-saving design. For developers running DPO, KTO or GRPO at scale, this is more than housekeeping: some distributed jobs may previously have produced incorrect gradients without making a dramatic fuss about it.

Hugging Face Watch analysis

What happened

The release removes trl.losses and routes DPO, KTO and GRPO through TRL’s existing chunked log-probability function. The project says this avoids maintaining duplicate versions of each trainer’s loss maths, which had allowed behaviour to drift between paths.

The release notes identify several concrete problems: KTO gradients were not always all-reduced under plain DDP, KTO could ignore the reference model under PEFT, and options including KTO class weights and DPO label smoothing were silently dropped. The new path streams selected-token probabilities without materialising full logits, so the memory benefit remains.

Key findings

  • Distributed training fix
    KTO now runs through the distributed wrapper under plain DDP, allowing gradients to be reduced across ranks.
  • More trainer options
    DPO can use mixed loss types, f-divergences and precomputed reference log-probabilities; GRPO gains entropy and off-policy masking.
  • A useful memory result
    On the supplied H100 test, precomputed reference log-probabilities reduced peak memory from 7.05 GB to 4.83 GB, a 31% cut.
  • A small speed trade-off
    The inline-reference benchmark rose from 0.2243 seconds to 0.2457 seconds per step, or 9.5% slower, in the listed configuration.
  • A new in-tree kernel
    TRL’s fused Triton log-probability and entropy kernel is now shipped in the project rather than fetched from the Hub, and is enabled by default on CUDA, ROCm and XPU with a PyTorch fallback.

Why it matters

Silent distributed-training errors are the particularly nasty sort: the run completes, the charts look purposeful, and the gradients have quietly gone their separate ways. TRL’s changes reduce that risk while opening options that the previous fused implementation rejected.

The release also makes a practical case for keeping performance work close to the framework that owns it. Moving the kernel in-tree removes a runtime download, an unrelated version gate and the need for trustremotecode=True for this component.

Our read

This is a substantial developer release disguised as a version number. Teams using TRL’s DPO, KTO or GRPO trainers should inspect the breaking changes and rerun their own benchmarks, especially if they rely on plain DDP, PEFT or experimental trainers. The headline is not simply “faster”. It is fewer duplicated code paths and fewer ways for a training run to be confidently wrong, which is a rather good bargain even when one benchmark gets 9.5% slower.

What to watch

  • Whether downstream projects still import the removed trl.losses classes.
  • How the new chunked path performs on larger models and multi-node jobs.
  • Whether the in-tree Triton kernel behaves consistently across CUDA, ROCm and XPU.
  • Which of the removed experimental trainers return through stable alternatives. Sources and evidence: Hugging Face TRL’s v1.14.0 release notes, published 25 September 2026. The performance figures and bug descriptions above are claims and measurements reported in those release notes.

Discussion spark: Should training frameworks prioritise one well-maintained implementation, even when a specialised path can be faster, or is that trade-off too costly for large-scale model training?

Sources and evidence
  • v1.14.0 (25 September 2026, 06:40 UTC)

not affiliated with or endorsed by Hugging Face

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.