Ivanti says its AI security tool invented patch details, so humans gate every briefing

Ivanti says a Claude-based tool used to prepare its enterprise patch guidance repeatedly invented details, including release patterns and product-edition information. The company now keeps every output behind human approval, because a polished vulnerability ranking is not much comfort if the model quietly misplaces the fire exit.

Watch Desk analysis

What happened

Ivanti executive Chris Goettl told VentureBeat that he spent months training the skill on vendor advisories and the company’s own spreadsheets. It does not access customer data, but during development it repeatedly produced unsupported details about Adobe’s release cadence and conflated Microsoft Office edition families.

The system is now part of Ivanti’s monthly Patch Tuesday workflow. Ivanti says replay tests against two months of previously hand-built work reached 98% alignment, while a task that once consumed about 48 combined staff hours ran in under 30 minutes with a reviewer. August was its first production month and September its second.

That speed claim comes with an important boundary. VentureBeat did not independently audit the dataset, scoring method or error categories, and independent security adviser Kayne McGladrey warned that reviewing generated rows may not reveal a vulnerability the system omitted altogether. The full account is in VentureBeat’s investigation of Ivanti’s AI-assisted patch guidance.

What to do now

• Ask where AI sits
  Establish whether deterministic software or a language model parses, counts and prioritises the vulnerabilities.
• Test for missing rows
  Validation should check source coverage, not merely whether the generated entries look plausible.
• Demand named review
  Find out who approves the guidance and whether that review chain is disclosed before priorities reach your queue.

Why it matters

Patch guidance drives decisions about which vulnerabilities receive attention first. September’s published Patch Tuesday totals varied widely between trackers, although VentureBeat notes that scope and counting methods can produce differences without any AI failure. Adding a language model creates another layer whose mistakes may alter the risk tier customers act on.

Ivanti’s disclosure is therefore more useful than a spotless marketing claim. It shows both the attraction and the weakness of the workflow: dramatic time savings are possible, but human review must be designed to find absences and unsupported claims, not simply admire a tidy spreadsheet.

Our read

Ivanti deserves credit for describing the fabrications and its approval gate. Buyers should now ask every security-guidance provider the same questions, especially what the output was validated against and whether the test can detect a missing vulnerability. “A human looked at it” is a start, not an evaluation method.

What to watch

• Whether Ivanti publishes measured omission and false-priority rates from production runs.
• Whether other security vendors disclose where generative AI enters their patch-ranking pipelines.
• Whether validation practices test complete source coverage as well as generated-row accuracy.
• Whether customers or regulators begin requiring written provenance for AI-assisted security guidance.

Discussion spark: What evidence would you require before trusting an AI-assisted vulnerability ranking in a production patching workflow?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.