AI-written tests are not the whole quality problem, argues TestMu co-founder

AI coding tools can produce tests quickly, but a passing test suite does not prove that software meets its requirements. TestMu AI co-founder Mudit Singh argues that teams need to separate regression testing from checking how a finished product actually behaves for users.

Watch Desk analysis

What happened

In a Forbes Technology Council article, Singh says AI-generated tests can preserve known behaviour while still sharing the coding assistant’s mistaken assumptions. His proposed distinction: regression tests check whether a change broke something that used to work; verification checks the running product against what the requirement asked it to do.

Singh recommends auditing customer-reported defects to see which regression tests could have caught them, reporting workflow verification separately from test pass rates, and judging AI-generated tests by whether they assert the ticket’s requirement. For critical journeys such as sign-up, checkout and billing, he argues for checks in real browsers and on real devices, ideally before a pull request is opened.

Why it matters

AI-assisted coding can increase the pace of changes without making a green dashboard any better at spotting a requirement that was misunderstood. If the same mistaken assumption shaped both the code and its test, the test may confidently approve the wrong thing. A check based on the requirement, rather than the implementation, has a better chance of catching that gap.

This is a practitioner’s argument, not a neutral audit of testing practice. Singh is a co-founder and growth lead at TestMu AI, a testing-platform company, so his advocacy for real-browser verification also aligns with his industry’s interests.

Our read

The useful takeaway is not to bin automated tests; it is to stop asking them to prove more than they can. Tests can guard against known regressions, while separate checks ask whether an important user journey works as intended. That distinction is less glamorous than another dashboard, but rather more likely to help when the dashboard is green and the customer is not.

What to watch

• Whether teams report user-workflow verification separately from test pass rates.
• Whether checks based on written requirements catch defects that self-generated tests miss.
• Whether the proposed approach improves outcomes without making review and release processes unwieldy.

Discussion spark: Should teams treat passing regression tests as enough to ship, or require separate checks of critical user journeys against the original requirements?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.