When an AI system starts slipping, teams need more than a dashboard showing that something has gone wrong. Forbes Technology Council contributor Hiren Dhaduk argues that production teams should test proposed changes against the same representative cases and agreed thresholds before release.
Watch Desk analysis
What happened
Dhaduk, CTO and co-founder of AI engineering company Simform, describes a production system whose quality deteriorated after months of stable performance. The team could see the decline, but had no shared set of examples to compare the last-known-good version with a proposed fix. That left engineers making changes without reliable evidence that they had improved the behaviour that mattered.
His proposed evaluation framework has three parts: a maintained set of representative examples, a consistent way to assess performance, and thresholds for deciding whether a change is ready to ship. Production failures should be added to the set, so future versions have to show they do not repeat them. Some checks can be automated; others need human review against defined criteria. Read Dhaduk’s full argument.
Why it matters
Monitoring can tell a team that a live system is struggling. Evaluation helps answer a different question: does the next version perform better on the cases the team cares about? That distinction matters when a change improves one measure but creates more retries, escalations or manual correction elsewhere.
Dhaduk argues that evaluation should sit inside the release process, with production failures feeding back into the test set. It is practical advice for teams trying to improve AI systems without relying on the traditional engineering method of shipping a change and hoping the graph looks cheerful.
Our read
The strongest point is the feedback loop: real failures become repeatable tests, and proposed fixes have to earn their place before release. Dhaduk’s piece is practitioner commentary, not a comparative study showing how much this approach improves outcomes. Still, teams can put the central question to work now: before changing a system, decide what evidence would show that the change helped.
What to watch
- Whether teams add meaningful production failures to their evaluation sets.
- Whether release thresholds cover critical behaviours, not just an overall score.
- Whether a change reduces the full cost of completing work, including retries and human review.
Discussion spark: Should AI teams block a release whenever a critical behaviour regresses, even if the system’s overall evaluation score improves?
Sources and evidence
- The Evaluation Framework: The Most Underbuilt Part Of An AI System – Forbes (5 November 2026, 16:45 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.