Microsoft researchers say fewer than 10 altered neurons can backdoor an AI agent

Microsoft researchers describe a way to make an AI agent execute an attacker’s command by altering fewer than 10 dormant neurons in its underlying model. The finding makes model-weight integrity a practical security concern, especially when an agent can use tools or a terminal.

Microsoft AI Watch analysis

What happened

In a research post published on 10 November, Microsoft describes “Lazarus”, a backdoor technique that needs no retraining. The researchers say a modified model can behave normally and pass standard evaluations until a hidden trigger appears. One example uses a date change; when triggered, the agent emits a command that downloads and runs code with whatever permissions the agent has.

The researchers say the technique can use reprogrammed neurons to detect a trigger and inject tokens into the model’s output. They also report that frontier AI agents can identify anomalous weight changes, but stress that scanning alone cannot prove a model is safe. Reliable detection, they say, depends on comparing weights with the original, unmodified model.

Why it matters

An agent’s permissions are only part of its security boundary. If its model weights have been tampered with, a model that appears to behave normally in routine tests could still produce a harmful command when a particular trigger occurs. That makes knowing where a model came from, and whether its weights changed on the way to deployment, more than tidy supply-chain paperwork.

Microsoft says its Foundry service ties onboarded open-weight models to their publishers and scans the surrounding software environment. It says it is also developing weight-scanning and red-teaming tools, and exploring cryptographic signing and tamper-evident audit trails. Those are Microsoft’s described safeguards and plans, not proof that any scan can guarantee safety.

Our read

The important point is not that open-weight models are uniquely vulnerable: Microsoft says the attack requires write access to a checkpoint, so internally developed models could also be exposed through a compromised account, pipeline or storage system. Treat model weights as security-sensitive software, limit what agents can do, and keep a trusted original for comparison. The humble “where did this file come from?” question has acquired a rather sharper edge.

What to watch

• Whether Microsoft publishes further technical detail or independent evaluation of the technique.
• How its proposed weight scanning and model-signing measures work in practice.
• Whether model providers and deployers adopt checks against trusted original weights.
• How agent permissions limit the damage if a model is compromised.

Discussion spark: Should organisations require a verifiable comparison with trusted original model weights before letting AI agents use sensitive tools, or is that too much friction for a threat that still needs write access to a checkpoint?

not affiliated with or endorsed by Microsoft