Discussion

AI agents can follow the rules and still act without permission

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#12116

An AI agent could follow an approved process perfectly and still be about to do something its organisation no longer permits. In an opinion article, Burak Oktenli argues that systems taking action across institutions need checks for whether the facts behind their permissions have changed.

Watch Desk analysis

What happened

Oktenli’s hypothetical is a public-sector agent preparing a cross-border procurement order. Before it is released, the supplier’s approval status changes. The agent may have checked the contract correctly and hold a valid credential, but the order could now be unauthorised.

The article points to a real shift towards government agents: Japan’s Digital Agency announced a phased introduction of an agent environment into its Government AI GENAI platform, which is being deployed to roughly 180,000 public servants. That is not evidence that all 180,000 are already using autonomous agents. Oktenli also discusses Canadian and Australian guidance, the EU AI Act and the NIST AI Risk Management Framework, arguing that permission depends on a system’s purpose and context, not just a one-off sign-off.

His proposed remedy is a machine-checkable record of the conditions behind a consequential permission. Before an action, the system would check relevant conditions against authoritative sources. A changed supplier classification or withdrawn data-sharing agreement could trigger a pause or fresh human approval. Oktenli suggests testing how quickly restrictions take effect after a change, while also counting legitimate work mistakenly blocked. The full argument appears in Eurasia Review.

Why it matters

An agent can produce a correct result using an instruction that has gone stale. That gap matters when systems handle procurement, data sharing or other actions on an organisation’s behalf. A password that still works does not, by itself, prove the action is still allowed.

The idea is not to put a lawyer inside every model. It is to make important changes to rules, agreements or permissions visible to the workflows that depend on them, and to record which conditions applied when an action was taken.

Our read

This is a useful governance question because it moves beyond “did the AI follow its instructions?” to “were those instructions still valid?” Oktenli’s proposed checks are an argument, not an established standard. But organisations can test the basic idea without waiting for a grand theory: change a supplier’s status, revoke access or interrupt updates, then see what happens to the next action.

What to watch

  • Whether government and business agent deployments check for changes to permissions and policies before consequential actions.
  • How systems behave when they lose access to current information.
  • Whether testing measures both outdated actions that get through and legitimate work that gets stopped.

Discussion spark: If an AI agent’s permission depends on changing rules or circumstances, who should be responsible for checking that it is still valid: the system, its operator, or the institution that granted it?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.