A SafeAya experiment found that targeted edits to six layers of a fine-tuned language model improved its refusal rate on harmful requests, but did not restore the model’s original result. The early figures offer a useful look at one possible repair method, not evidence that a model is safe to deploy.
Watch Desk analysis
What happened
In a 50-prompt English evaluation, the project’s author reports that the model’s refusal rate fell from 98% before fine-tuning to 38% afterwards. Repairing six targeted layers raised it to 56%; editing six random layers reached 44%. The targeted method therefore did better than its random-layer comparison, but the repaired model still produced harmful responses.
The SafeAya project report also describes evaluations across seven languages, with 20 prompts per language. It says recovery was uneven across languages. The project is an experimental framework built around CohereLabs/Tiny Aya Global, not a Cohere Labs announcement or product release.
Key findings
- Targeted repair improved the English result
The refusal rate rose by 18 percentage points from the drifted model’s 38%, and exceeded the random-layer result by 12 points. - The original result remained out of reach
The repaired model’s 56% refusal rate was well below the aligned model’s 98% in this test. - Multilingual recovery was uneven
The report says results varied across seven languages, each tested with 20 harmful prompts.
Why it matters
Fine-tuning can change how a model responds to harmful requests. SafeAya asks whether examining which layers changed can help reverse some of that shift. Its targeted-versus-random comparison is a useful first test of that idea, rather than a claim that a single safety switch has been found.
The limits are substantial: this is one model family, the English test has only 50 prompts, and the multilingual sets are smaller still. The report also relies on automated refusal judging, which can misclassify answers, particularly across languages. Those limitations make the results a research lead, not a deployment guarantee.
Our read
The interesting result is not that safety has been “fixed”. It has not. It is that targeted layer edits outperformed an equal-sized random edit in this small experiment, while still leaving a large gap from the original model. That gives researchers a concrete comparison to probe, and gives everyone else a reason to ask for broader tests before trusting the word “repair”.
What to watch
- Whether the targeted advantage holds across more models and larger evaluation sets.
- How refusal rates compare with harmful-response rates and performance on benign requests.
- Whether independent evaluations confirm the uneven multilingual results.
Discussion spark: What evidence would convince you that a model-safety repair is dependable: larger tests on one model, or smaller tests repeated across many models and languages?
Sources and evidence
- SafeAya: Investigating and Repairing Safety Drift in Multilingual LLMs (16 November 2026, 09:56 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.