Discussion

Capgemini warns that AI agents may be right individually and wrong together

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#15003

As businesses move from single AI agents to systems of agents, Capgemini argues that the challenge is governing how they work together. Its proposed answer is oversight at the level of the whole enterprise, not just checks around each individual agent.

Watch Desk analysis

What happened

In an article published on 20 November, Capgemini executives set out their expectations for AI in 2027. Deputy Group CTO Sudhir Pai says existing systems are not designed to test and govern growing networks of agents. “Every agent can be right, while the system is wrong,” he warns, pointing to the possibility of serious business consequences.

Pai proposes three layers of control: a harness around each agent, a control plane for each agentic system, and an enterprise control plane across them. The latter could track agents and their permissions, monitor outcomes and consumption, show how agents interact, limit what they can do together and provide a way to stop them. The article also discusses AI-enabled cyberattacks, agentic research and commerce, presenting these as forecasts and industry perspectives rather than measured findings. Read the article.

Why it matters

An agent can follow its own rules and still contribute to a system that behaves in ways its operators did not intend. That makes the connections between agents, their combined permissions and their overall impact a practical governance problem, not simply a question of whether each agent passes an individual test.

Capgemini’s proposed control layers offer a concrete way to frame that problem: track what is deployed, see how systems interact and retain the ability to limit or stop them. The article is a company perspective, not proof that these controls will prevent failures. But it identifies the awkward gap between testing one agent and knowing what a larger network will do.

Our read

“Every agent is behaving correctly” is cold comfort if the whole system is behaving badly. Capgemini’s useful contribution is to make system-level oversight more specific: know which agents are operating, what they can access, how they affect one another and where intervention is possible. That is a more grounded discussion than assuming that adding more agents automatically adds more capability.

The proposed controls deserve scrutiny alongside the capabilities they are meant to govern. A register and a stop button sound sensible; their value depends on whether operators can see interactions and act before consequences become difficult to reverse.

What to watch

  • Whether enterprise platforms provide the agent inventories, permissions and interaction monitoring Pai describes.
  • How organisations test multi-agent systems, rather than only testing agents individually.
  • Whether future deployments show that system-level controls work in practice, including when human oversight cannot sit in every loop.

Discussion spark: If individual agents pass their tests but the wider system behaves unpredictably, should organisations be allowed to deploy it at scale?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.