Microsoft has introduced run-assert-eval, a VS Code skill intended to help developers discover risks in AI agents, test them and enforce policies while those agents are running. The important shift is practical: runtime governance is being folded into the development workflow, rather than left as a document someone promises to revisit later.
Microsoft AI Watch analysis
What happened
Microsoft’s Command Line team says run-assert-eval combines three components: Clarity for threat modelling, ASSERT for generating evaluations, and the Agent Control Specification for testing and enforcing runtime policies. The tool is presented as a single loop covering risk discovery, evaluation and policy enforcement.
The release is aimed at AI-agent development in VS Code. Microsoft says the approach is designed to preserve comparison integrity while developers test whether an agent follows the policies imposed on it. The supplied announcement does not provide independent benchmark results, supported agent frameworks, pricing or details of how much developer intervention the workflow requires. Those omissions matter, but the release itself is concrete enough to examine.
Why it matters
Agents are moving from answering questions to using tools and taking actions. That makes a failure harder to spot after the fact, particularly when an agent has access to business data, external services or sensitive operations. A workflow that can identify risks, generate tests and enforce controls during execution could give builders a more repeatable way to catch bad behaviour before deployment.
It also reflects a welcome change in emphasis. “Responsible AI” is often presented as a principle; run-assert-eval tries to turn it into something closer to a test suite. The less glamorous machinery may be the part that decides whether agentic software becomes dependable or merely enthusiastic with permissions.
Our read
This is a promising development-tool release, not proof that Microsoft has solved agent safety. The useful question is whether developers can reproduce its checks, inspect the policies being enforced and see meaningful failures before an agent reaches users.
Microsoft is betting that safety works best when it lives beside the code. That is probably right. But a safety loop is only as good as the threats it models and the tests it runs. A tidy green tick can still be wrong, which is the sort of plot twist software teams could do without.
What to watch
- Whether Microsoft publishes examples of failed evaluations and the fixes they triggered.
- Which agent frameworks, tools and model providers the skill supports.
- How the Agent Control Specification handles permissions and irreversible actions.
- Whether independent developers can audit or reproduce the evaluations. The release is covered in Microsoft’s run-assert-eval announcement.
Discussion spark: Should runtime safety checks be mandatory for AI agents with access to company systems, or would that slow development without reliably preventing failures?
Sources and evidence
- Introducing run-assert-eval: Find the risk, fix it, prove it (24 September 2026, 17:28 UTC)
not affiliated with or endorsed by Microsoft