Raindrop says giving AI agents lessons from past tasks improved their customer-support benchmark results without retraining. Adding the agents’ reasoning to those lessons raised the average resolution rate further, though the size of that extra gain varied sharply by model.
Raindrop Watch analysis
What happened
In a 9 November blog post, Raindrop describes testing four open models on telecom support tasks in τ²-bench. Agents received a sequence of tickets and used account and phone tools to diagnose problems. A reviewer model updated each agent’s instructions after every ticket, either from its actions and outcomes alone or with its reasoning included as well.
Across the four models, the average share of tickets resolved rose from 33.5% with no memory to 59.2% with action memory and 66.5% with reasoning memory. The extra benefit was uneven: reasoning memory added 16.7 percentage points for DeepSeek v4 Flash, but just 0.4 points for MiniMax-M2.5. Raindrop says reasoning memory cost about 30% more per ticket than action memory in its experiments.
Why it matters
The result suggests agents may improve at repeated tasks by updating their instructions from experience, rather than changing the underlying model. It also points to a practical choice: recording actions and outcomes delivered most of the average improvement, while retaining reasoning brought a smaller additional gain at higher cost.
The benchmark has important limits. Agents worked through simulated telecom tickets without a simulated customer or troubleshooting guide, and an automated grader supplied clear success signals. Raindrop says real-world feedback is noisier, so these results do not establish that the same gains will carry over to production support.
Our read
The useful finding is not simply that memory helps. It is that action-based lessons did much of the work, and the value of adding reasoning depended on the model and task. Teams considering agent memory should compare both approaches on their own workflows, including the extra cost, rather than assuming more context is always better. A benchmark can give a clean answer; customers, less so.
What to watch
- Whether similar gains appear when agents learn from messier real-world feedback.
- Whether reasoning memory’s extra cost is justified on tasks where it adds little.
- How results change when agents can consult customers or use the benchmark’s troubleshooting guide.
Discussion spark: Would you pay around 30% more for an agent to learn from its reasoning, or is action-and-outcome memory enough for most support tasks?
Sources and evidence
- Source update (9 November 2026, 00:00 UTC)
Independent WittyWires tracker for public updates about Raindrop. Not affiliated with or endorsed by Raindrop; this is not an official account.