GPT-6 Luna averaged just $0.007 per workplace situation in Deepnote Research’s Agent Initiative Benchmark, but scored zero in nine of 50 runs. That gap between bargain pricing and unpredictable behaviour matters for any team considering an agent that watches workplace messages and decides when to speak.
Watch Desk analysis
What happened
OrcaRouter’s account of the benchmark says models were placed in a simulated company and asked to read Slack messages, investigate where needed, and decide whether to reply. Forty of the 50 scenarios called for a message; ten were traps where the right move was to stay quiet.
GPT-6 Luna scored 6.38 out of 10, sent a message in 38 of the 40 cases where one was needed, and averaged $0.007 per task. Claude Fable 5.1 scored 7.50 at $3.02 per task. The report says the current Claude Fable 5.2 scored 8.17 at $1.87 per task, making the older-model comparison less useful for anyone choosing today.
The striking wrinkle is consistency: Luna recorded nine perfect scores and nine zeros. The benchmark also found it intervened in six of the ten situations where the issue had already been handled. Deepnote describes the test as an early measurement based on 50 situations and says a larger task set is planned.
Why it matters
A low per-task price could make it practical to let an agent monitor more conversations. But a bot that sometimes misses the moment and sometimes speaks when nothing needs doing can wear down trust in the messages that are worth reading. The cost of an unnecessary intervention is not only the model call; it is the channel’s attention.
The results also resist a tidy leaderboard verdict. The report says several models’ scores fall within overlapping uncertainty intervals, and Luna’s nine perfect runs sit alongside nine failures. Cheap is a useful property. It is not the same thing as dependable.
Our read
This is a worthwhile test of a harder question than “Can the model answer?”: can it judge when a workplace needs an answer at all? Luna’s low reported cost makes further experiments plausible, but these results are a reason to test silence, timing and consistency, not just the quality of messages it does send. The benchmark is a first measurement, not a verdict on every team’s Slack.
What to watch
- Whether the larger task set changes the reported score spread and failure rate.
- How Luna performs at its default effort setting, since this test used high reasoning effort.
- Whether teams can reduce unnecessary interventions without losing useful ones.
Discussion spark: For a workplace agent, would you prioritise a very low cost per task or predictable judgement about when to stay silent?
Sources and evidence
- GPT-6 Luna: $0.007 a Slack Thread, 430x Under Fable 5.1 – OrcaRouter (13 November 2026, 19:13 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.