A developer has outlined a reporting system where code checks claims about data before they reach the page, rather than asking a language model to be right. In tests on generated datasets, the approach answered 92 of 94 questions correctly, while leaving an important gap: checking whether statements are true does not guarantee a report is fair or complete.
Watch Desk analysis
What happened
The system divides the job between software and a model. Code calculates and renders numbers, and supplies a catalogue of permitted claims, such as trends, rankings and comparisons. The model chooses and arranges those claims, but must cite their IDs rather than inventing them. A word guard also rejects unsupported language such as “increasing” or “highest” unless it appears in a checked claim.
The author reports that the system was tested on three generated datasets, covering retail sales, card payments and HR. In the version with a clarification rule for questions that name no metric, 92 of 94 answers matched the expected result. The system traced every number correctly in all runs; its misses involved choosing the wrong query or outcome. For narration, code wrote six of 72 sections after model drafts failed checks. The author says the evaluation ran once, and measures routing rather than the quality of the writing. Read the Hugging Face community article.
Why it matters
A sentence can mislead without getting a single number wrong. If sales rose overall but fell in one region, a report that mentions only the rise can pass arithmetic checks while still giving readers the wrong impression. This design makes individual claims traceable, but its author explicitly says it cannot establish that a report is fair in general.
That distinction is useful well beyond dashboards: AI-generated explanations often sound more settled than the evidence underneath them. Here, the proposed safeguard is architectural. The model helps decide what to say, while software controls which data claims can be published and shows readers their provenance.
Our read
The promising idea is not that a small model has been made infallible. It is that the system gives the model less room to smuggle an unchecked interpretation into fluent prose. The test is an encouraging early result, not a verdict: it uses generated data, ran once, and does not assess whether the final explanations are good or balanced.
What to watch
- Whether the approach is tested repeatedly and on real-world datasets.
- Whether readers find claim provenance useful enough to spot omissions as well as errors.
- Whether larger and smaller models change writing quality without changing which claims pass the checks.
Discussion spark: Would you trust a data report more if every factual claim were code-checked, even if the system still could not guarantee that the report was fair or complete?
Sources and evidence
- Code-Checked Narration: small models write the report, code checks every claim about the data (14 November 2026, 16:30 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.