TRL 1.17 adds single-node fine-tuning for Mistral Large 4 and changes AsyncGRPO defaults

Hugging Face’s TRL 1.17 release adds a recipe for fine-tuning Mistral Large 4 on one eight-GPU node, alongside substantial changes to reinforcement-learning training. For practitioners, it offers new model support and performance improvements, but also changes an AsyncGRPO default in ways worth checking before an upgrade.

Hugging Face Watch analysis

What happened

TRL’s v1.17.0 release notes describe a LoRA fine-tuning example for Mistral Large 4, a one-trillion-parameter model with 52 billion active parameters. The example uses eight B300 GPUs and FP8 weights; the notes report 32,700 tokens per step in about eight seconds, with 142 GiB of memory per GPU. Those are release-note measurements for the stated setup, not a general speed guarantee.

The release also adds AMD Instinct support through PyTorch’s ROCm build, including an AMD CI job, and extends tree packing to several gated-delta-network hybrid models. In AsyncGRPO, score centring with masked importance sampling is now the default. The notes say this can address a mismatch between FP8 sampling and bf16 training, while costing 15 to 20 per cent of generation throughput.

Why it matters

The Mistral example makes a very large model more approachable for teams exploring fine-tuning, while ROCm support gives AMD GPU users a clearer route into TRL training. The update is not just a model-compatibility checklist: its training changes affect how some jobs behave by default.

In particular, the new AsyncGRPO settings drop tokens whose sampling ratios fall outside the stated range. Developers who depend on the previous clipped objective can set scorecentering=False; those using vLLM importance-sampling correction also need topk when using topp or minp filters. That is the sort of release-note detail best discovered before a long run, rather than after one.

Our read

This is a substantial release for people building and training language models, with useful gains in model coverage, hardware support and rollout efficiency. The Mistral figures are a concrete starting point, not a promise that every setup will match them. Our advice: read the changed defaults, check the required dependency versions and test the new configuration on a representative run before moving important training jobs across.

What to watch

• Whether users reproduce the Mistral fine-tuning results on comparable hardware and settings.
• How ROCm support performs across real training workloads beyond the new CI coverage.
• Whether the AsyncGRPO defaults improve training outcomes enough to justify their stated throughput cost.

Discussion spark: Should TRL make the new AsyncGRPO objective the default, or should a change that trades throughput for training stability require users to opt in?

not affiliated with or endorsed by Hugging Face