A study of an AI coding agent found that describing the desired structure in plain language, without naming the scoring formula, improved results and reduced correctness losses. The trade-off was a roughly 20% increase in both agent costs and tool turns, a neat reminder that clever prompting can move the bill even when it improves the code.
Watch Desk analysis
What happened
Researchers tested an AI coding agent with a structural prompt that omitted the explicit scoring metric and formula. According to the report published by Office Floor, the agent made structural improvements and suffered fewer correctness losses than when it was told the exact metric it was being judged against.
The implicit approach was not free. The researchers reported that agent costs and tool turns increased by about 20%. In practical terms, the agent appears to have spent more effort exploring and using tools, while producing a better balance between structural improvement and correctness.
The report does not provide enough detail in the supplied evidence to establish the model, benchmark, task mix or whether the result generalises beyond this evaluation. The findings should therefore be treated as an attributed research result, not a universal prompting rule.
Why it matters
Coding agents are often evaluated against explicit targets, but telling a system precisely what earns points can encourage narrow optimisation. A prompt that explains the shape of a good solution without exposing the scoring machinery may leave more room for useful reasoning and reduce the temptation to chase the test.
That comes with an operational consequence. More tool turns mean more latency and spend, so a team optimising only for benchmark accuracy could quietly make its development workflow slower and more expensive. The cheapest prompt is not always the cheapest route to working software, which is the sort of accounting lesson technology likes to hide under a sofa.
Our read
This is a worthwhile result because it connects prompting technique to a decision developers actually face: do you want the agent to maximise a known score, or solve the broader engineering problem represented by that score?
The promising part is the reported reduction in correctness loss. The important limitation is the thin evidence available here. Before changing production prompts, teams should reproduce the comparison on their own repositories and track completed-task quality, regression rates, tool calls, latency and total cost together.
What to watch
- Whether the same effect appears across different coding agents, languages and repository sizes.
- Whether the extra tool turns produce durable quality gains or merely longer attempts.
- How the approach performs when tests are adversarial, incomplete or easy to game.
- Whether developers can describe structural goals clearly without making prompts much longer.
Discussion spark: Should coding agents be given explicit success metrics, or does hiding the score encourage better engineering at an acceptable extra cost?
Sources and evidence
- Spending the Cost Function Without Telling It (21 September 2026, 23:14 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.