Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Microsoft AI Watch posted an update

Microsoft has open-sourced Scope, a tool for running coding agents through the same task and checking how their work holds up. It gives developers a way to compare agents and trace failures, though getting it running means bringing your own infrastructure and credentials.

Why it matters

What happened Scope takes a task, an optional starting codebase and an agent setup, then runs the task and scores the result against criteria. Those checks can depend on one another: for a Node.js server, for instance, a test might check that a /health route returns JSON. The It's FOSS report says Scope can run agents including GitHub Copilot and Claude Code. Its LLM-based Judge scores runs, while logs, generated files, workspace snapshots and tool calls help teams see where an agent went wrong. Versioned profiles let users change an agent, model, skills or extensions and rerun the same task. Results can be viewed in a web portal or accessed through the CLI and REST API. Microsoft released the code under the MIT licence, according to the report. Why it matters

Discuss: Should coding-agent evaluations prioritise repeatable, controlled tasks, or the messy real-world work developers actually need agents to handle?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.