A Qwen model improved on coding benchmarks after Hugging Face trained small adapters inside three different AI-agent harnesses. The results suggest that tuning a model for the tools and tasks it will actually face can improve performance while using fewer tokens.
Hugging Face Watch analysis
What happened
Hugging Face trained a separate LoRA adapter for Qwen3.8-27B inside each of the Pi, OpenCode and Hermes Agent harnesses, then combined them into a single adapter that works across all three. The company says each adapter beat the unadapted model under its own harness on both performance and efficiency benchmarks.
One example: on Terminal-Bench 2.1 using Pi, the model’s pass@1 score rose from 39% to 55%, while average token use per task fell from 2,900 to 1,500. Hugging Face describes the training recipe and results, including its task-selection and validation process.
Why it matters
Agent performance depends on more than the model. The harness determines how it uses tools, and the tasks and feedback used in training shape what it learns to do. Training inside the actual harness may therefore close some of the gap between a general-purpose model and one tuned for a particular workflow.
The token reduction is useful, but the reported benchmark gains are not a promise that every coding agent will get the same lift. The results come from Hugging Face’s experiments with a particular model, training recipe and set of tasks.
Our read
This is a concrete case for tuning the whole agent setup, not just swapping in a bigger model and hoping the tools behave. The most interesting wrinkle is that Hugging Face says one merged adapter can work across three harnesses, rather than requiring a separate tuned model for each. Developers can explore the published recipe and judge whether its approach fits their own tasks. A benchmark win is a good start; reproducibility is the bit that earns the second cup of tea.
What to watch
- Whether independent teams reproduce the reported gains on their own coding tasks.
- How performance and token use change across models and harness versions.
- Whether the merged adapter holds up beyond the benchmarks described.
Discussion spark: Would you rather use one adapter that works across several agent harnesses, or tune a separate one for each workflow?
Sources and evidence
- Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost (20 November 2026, 00:00 UTC)
not affiliated with or endorsed by Hugging Face