A Hugging Face researcher building an open version of Open Pangram has paused to investigate suspiciously strong results from an early AI-text detection experiment. The useful news is not a breakthrough score, but a public example of how quickly a plausible evaluation can go wrong when the data and task are narrower than intended.
Watch Desk analysis
What happened
In a community article published on 25 September, River described an attempt to train an Open Pangram-style classifier using a causal language model, with Qwen among the proposed bases. Early experiments used document collections including PMC and ACL papers, alongside EditLens text, and compared methods ranging from character and word n-grams to transformer-based models.
The first training run produced a mediocre result at first, with reasonable recall but roughly 4% false positives. After the decision threshold was changed, the reported results became suspiciously high. River said the ROC charts also raised concerns and concluded that something had gone wrong in the exchange with the AI assistant helping the project.
The immediate diagnosis was a scope problem: the assistant had focused on doing well on papers, while the intended goal was broader detection across different document types. River now plans to widen the dataset using labelled examples of human and AI-written text, and to tune the training process so results improve consistently and vary less between random seeds.
Why it matters
AI-text detection is unusually vulnerable to flattering tests. A model can appear impressive when it learns quirks of a narrow source, genre or dataset rather than a durable distinction between machine-written and human-written text. A detector trained mainly around papers may therefore struggle the moment it meets emails, reports, posts or edited prose.
This is also a useful reminder that an AI assistant can accelerate an experiment without reliably understanding what the experiment was meant to prove. The glamorous part is producing a chart. The expensive part is noticing when the chart is answering the wrong question.
Our read
River’s willingness to stop, question the attractive numbers and publish the awkward intermediate result is more valuable than another confident detector leaderboard. The project has not established that its model works, but it has identified a concrete evaluation failure and a sensible next step: broaden the data, inspect the labels and repeat the test across seeds and document types.
For readers assessing AI-detection claims, the practical takeaway is simple: ask what kinds of text were tested, how the labels were made and whether the result survives outside the source material that shaped the model.
What to watch
- Whether the expanded dataset includes varied human and AI-written material rather than mostly academic papers.
- Whether performance remains stable across random seeds and previously unseen document types.
- Whether the project publishes threshold choices, false-positive rates and evaluation splits.
- Whether the next results still look unusually strong after the suspected pipeline problem is fixed.
Discussion spark: Should AI-detector projects publish failed evaluations and suspiciously good intermediate results as standard practice, or would that mostly create more noise around tools that are already hard to assess?
Sources and evidence
- DIY Open Pangram – Part 1 (25 September 2026, 06:27 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.