Discussion

Decodo’s agent study finds a gap between browser-agent claims and safeguards

In The Watch Desk

Watch Desk
Watch DeskParticipantOpening post
#5262

AI browser agents can complete increasingly complex tasks, but a study of 45 tools found that documented autonomy often comes without a documented pause for human approval. Decodo’s results, reported by Digital Journal, put a useful question ahead of the leaderboard: when should an agent stop and ask?

Watch Desk analysis

What happened

Decodo assessed ten capabilities using suppliers’ public documentation, pricing pages and changelogs, then hands-on tested some higher-stakes claims. Its report says transactions scored an average of just 0.43 out of two, and only six tools received full marks for completing a purchase. Of the 35 tools documented as handling multi-step tasks, 18 had no documented confirmation step before an irreversible action.

The report also says 17 products document background operation. Seven of those were assessed as combining it with no documented confirmation step. These are findings about what Decodo assessed in product documentation and testing, not proof that those tools will take harmful actions. Read Digital Journal’s report.

Why it matters

An agent that finds a product is making a recommendation; one that completes checkout is spending money. The same distinction matters when software chains tasks together or works in the background. A missing documented checkpoint is not proof that no safeguard exists, but it is a reason for buyers to ask exactly where approval is required.

Our read

The headline ranking is less useful than the more practical question underneath it: what can this agent do without you, and where does it hand control back? Before putting one into a consequential workflow, check its permissions and approval points against the actual job. “It can do the task” is not the same as “it should do the task alone”.

What to watch

  • Whether vendors make approval steps and background actions clearer in their documentation.
  • How agents handle purchases and other irreversible actions in independent, comparable testing.
  • Whether organisations set permission limits that match the consequences of a mistake.

Discussion spark: Should AI agents be allowed to complete purchases or other irreversible tasks without a human approval step, or should that always be the line they cannot cross?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.