Enterprise AI needs better tests than a leaderboard score

AI systems can ace public benchmarks and still fail the work a business needs them to do. Abaka AI research executive Zihan Wang argues that companies should test how models behave on their own data, when they are uncertain, and when a task calls for refusal.

Watch Desk analysis

What happened

In a Forbes Council article published on 5 November, Wang says public benchmarks such as MMLU and GPQA measure general capability, not whether a model is reliable on a company’s documents or in its workflows. He recommends three practical checks: test what the model does when it does not know; evaluate it on a private, held-out set based on the company’s own data; and use independent, adversarial evaluation rather than relying only on tests designed by the model’s builder.

Wang also argues that evaluation should continue after launch, using production samples and domain expertise to test for failures, including behaviour under distribution shift. His piece is an executive’s argument for how businesses should assess AI, not an independent study establishing how often models fail.

Why it matters

A benchmark score is tidy, comparable and wonderfully slide-friendly. But a company choosing a model needs to know whether it handles the company’s actual work, and whether it admits uncertainty instead of bluffing. Those are different questions.

Wang’s advice gives buyers something concrete to ask vendors for: performance on a relevant test set, evidence of refusal behaviour, and results from evaluations designed to find failures. It also shifts evaluation from a one-off launch hurdle to continuing work as systems and data change.

Our read

The strongest part of Wang’s argument is the focus on task-specific testing. A general leaderboard can help narrow a shortlist, but it cannot tell a buyer whether a model is fit for a particular job. The practical move is to define the failures that matter, test for them on held-out examples, and keep checking after deployment.

Wang’s article includes striking figures on hallucinations and AI-related losses, but the supplied text does not provide enough underlying detail to assess those numbers. The case for better evaluation does not need them: the distinction between general capability and performance on your own work is useful on its own.

What to watch

• Whether AI vendors provide results on customer-defined, held-out test sets.
• How buyers assess refusal behaviour and performance when data or tasks change.
• Whether independent, ongoing evaluation becomes part of routine AI procurement.

Discussion spark: Should companies require vendors to pass independent tests on their own data before deployment, or is that too much to ask when models and workflows keep changing?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.