A modified version of DeepSeek Harness used substantially fewer tokens per task in coding benchmarks, while its success rate also rose on the reported tests. The result suggests agent workflow changes can reduce model use without changing the underlying model, though the size and consistency of the gains vary.
DeepSeek Watch analysis
What happened
TokenPost reports on an 23 October preprint testing HarnessMM, a system that stores actions an agent has already chosen and can carry them out after a tool result arrives, when preset conditions are met. If a fresh decision is needed, the agent still calls the model.
The comparison used 98 HumanEval tasks and 257 MBPP-sanitized tasks. The changes to the harness included more than stored-action reuse, so the headline token reduction should not be read as the effect of that feature alone.
Key findings
- DeepSeek Harness on HumanEval
Average combined input and output use fell from 96,280 to 11,620 tokens per task; success rose from 71.43% to 85.71%. - Codex on HumanEval
Token use fell 29.54%, while success remained at 90.82%. - Results differed on MBPP-sanitized
DeepSeek Harness token use fell from 53,980 to 10,610, with success rising from 90.27% to 91.83%. Codex used fewer tokens, but its success rate slipped from 91.83% to 90.66%. - The reduction was mainly in input tokens
Output use for DeepSeek Harness rose from 3,830 to 4,530 tokens per task. The report says 147 of 539 fewer paid model calls were certified cases where stored actions replaced reasoning calls.
Why it matters
AI agents can spend heavily on repeated context and model calls, so reducing that workload could matter to teams running coding tasks at scale. But tokens are not the whole bill: the reported benchmarks do not establish dollar savings, shorter run times, lower energy use or production performance. Costs depend on provider accounting and how often an earlier action can safely be reused.
Our read
This is a promising result about agent design, not evidence that one clever cache makes coding agents dramatically cheaper in every setting. The important detail is that the whole harness changed, and the results differed by model and benchmark. Useful signal, with fewer reasons to declare victory than the headline number suggests.
What to watch
- Whether the preprint’s results are reproduced independently.
- How often real coding tasks can reuse stored actions without needing a new model decision.
- Whether later tests measure cost and speed in production, alongside benchmark tokens and success rates.
Discussion spark: Would you trust a coding agent to reuse an earlier action when a tool result meets preset conditions, or should it ask the model to reason again each time?
Sources and evidence
- Deepseek official source (2 November 2026, 09:23 UTC)
not affiliated with or endorsed by DeepSeek