NTT’s AI tools found 80% of planted software inconsistencies in an early test

NTT has developed tools to help AI make sense of older software, linking specifications, design documents, code and tests. In preliminary experiments, the tools found 33 of 41 deliberately planted inconsistencies and identified three previously unknown bugs, The Fast Mode reports.

Watch Desk analysis

What happened

NTT’s approach combines traceability analysis, which maps how software documents and code relate, with automated test generation to find gaps in existing tests. The company evaluated the tools on a sample application.

For inconsistencies, the system detected 33 of 41 planted errors, or 80%. A conventional retrieval-augmented generation approach detected 11, or 27%, in the same evaluation. In a separate test, the automated testing technology found four of six deliberately planted bugs and also identified three latent bugs that had not been planted.

NTT says it plans to test the technology on real software maintenance projects, expand the range of supported languages and artefacts, and develop its coverage further.

Why it matters

Generative AI is often shown writing new code. Maintaining old systems is a different, less glamorous challenge: documentation can fall out of step with code, and changes can break functions no one thought to retest. NTT’s tools are designed to connect that scattered information and generate tests for areas existing tests may miss.

The results are promising, but they come from preliminary experiments on a sample application, not a demonstrated deployment across large, live systems. The numbers are a reason to pay attention, not a guarantee that the same detection rates will hold in production.

Our read

The useful idea is not simply asking a chatbot to review legacy code. NTT is trying to give AI a structured map of how requirements, designs, code and tests fit together, then use that map to check for gaps.

That could make AI-assisted maintenance more dependable. The real test is whether it helps engineers find consequential faults in varied, working systems without burying them in noise.

What to watch

• How the tools perform on real maintenance projects, beyond the sample application.
• Whether NTT publishes results across different system sizes and programming languages.
• How many findings engineers judge useful, and how much checking the tools save them.

Discussion spark: Would you trust AI to help maintain legacy software on the strength of promising sample-app results, or should it first prove itself across real systems?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.