An AI safety researcher corrected the code, not just the conclusion

A Hugging Face researcher has withdrawn the central technical argument from an earlier analysis of DeepSeek-V4-Flash, after testing the model’s actual implementation on four H200 GPUs. The correction is valuable because it shows how easily an AI-safety result can go wrong when theory, reference code and the framework people actually run drift apart.

Watch Desk analysis

What happened

Timothy Rohrbaugh writes that an earlier post argued Manifold-Constrained Hyper-Connections made directional ablation fundamentally invalid. New experiments found the opposite: the model’s ordinary outputhiddenstates path exposed the full four-stream tensor, while the model’s learned projection, not a simple average, was the useful representation.

The new work reports that rank-one ablation still functioned across the model’s 43-layer stack, with a measured KL divergence of 0.147 and 12 of 83 outright refusals converted to deflections. Rohrbaugh also identifies a more consequential limitation: block-scaled FP8 prevented the intended edit from reaching most of the routed expert weights. The earlier analysis had been looking in the wrong part of the model.

Key findings

• The hook claim was inverted
  The framework exposed the full stream tensor, so the supposed instrumentation barrier was not real.
• Averaging was the real mistake
  The model’s learned collapse placed about 98.9% of its weight on one stream, making an unweighted mean a poor summary.
• The edit still worked mechanically
  Rank-one directional ablation survived across the stack, despite the earlier theoretical objection.
• Quantisation blocked the practical edit
  Routed experts accounted for 99.2% of the MLP write path but were not reached by the attempted intervention.
• The scoring system distorted results
  Truncation and language-specific character limits caused the refusal detector to misclassify outputs, hiding useful gradations such as refusal becoming deflection.

Why it matters

This is a story about AI research hygiene as much as one model. The correction says a single forward pass, printed tensor shapes and a check against the production framework would have overturned the original central claim before the theory grew elaborate.

It also gives readers a practical warning about benchmark results. If the instrument cannot distinguish a refusal from a truncated answer, or treats different scripts by the same character threshold, the optimisation target can quietly reward the wrong behaviour. The spreadsheet may look calm while the experiment is shouting.

Our read

The most impressive result here is not that the revised ablation worked. It is that the author published the failure in enough detail for others to avoid repeating it. AI safety needs stronger tooling, but it also needs a culture that treats retractions as useful technical output rather than reputational surrender.

The lesson is straightforward: inspect the tensors, inspect the deployed implementation and measure the scorer before trusting the score. Grand claims about model behaviour should have to survive those rather unglamorous checks.

What to watch

• Whether independent researchers reproduce the corrected ablation results.
• Whether future DeepSeek integrations expose the same tensor and hook surfaces.
• Whether refusal benchmarks report truncation, language and deflection errors separately.
• Whether routed expert weights can be edited reliably under block-scaled FP8.

Discussion spark: Should AI research papers and safety claims be expected to publish executable instrumentation checks alongside their theory, or would that create an unrealistic burden for researchers?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.