Researchers describe a technique called grafting that transfers changes from synthetic document fine-tuning on a base model into a model that has already been post-trained. They report testing it across five model families, up to 284 billion parameters, with more than a 50% average reduction in two kinds of damage associated with the usual approach.
Watch Desk analysis
What happened
In a post on LessWrong, the researchers describe fine-tuning a pre-training checkpoint, then adding the resulting weight difference to a post-trained model. The aim is to install behaviours learned from synthetic documents while retaining work already done during post-training.
The researchers say grafting maintained behavioural expression while easing losses in coherence and capability. They also report reducing reality drift and preference-coherence loss by more than 50% on average, with the changes installed at a strength comparable to standard post-training fine-tuning.
Why it matters
Synthetic documents can be used to shape what a model knows or how it behaves. But if that training damages coherence or shifts a model away from reality, the added capability comes with an awkward bill. Grafting is an attempt to keep the useful change while limiting that collateral damage.
The reported tests span five model families and reach 284 billion parameters, making this more than a result confined to one small model. Still, those figures describe the researchers’ experiments; the supplied account does not give enough detail to judge how broadly the method works in other settings.
Our read
This is a neat bit of model-training engineering: rather than asking teams to choose between new behaviour and the post-training they already value, grafting tries to combine them. The promising part is the reported reduction in coherence and reality drift. The next question is whether independent tests find the same gains when the method meets different models and training tasks.
What to watch
- Whether follow-up work reproduces the reported reductions across the five model families.
- How grafting performs on tasks beyond those described in the researchers’ account.
- Whether the method can be applied without sacrificing the capabilities post-training was meant to preserve.
Discussion spark: Would you trust a model-training technique on the strength of results across five model families, or wait for independent replication on your own use case?
Sources and evidence
- You can graft SDF changes from base models onto post-trained models (6 October 2026, 02:17 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.