Coding agents repeatedly send their full conversation history to a model, so small changes near the start of a prompt can force large amounts of work to be repeated. A new Ollama-hosted guide from Ollama and Together AI explains how developers can protect that cached work and keep long-running agent sessions more efficient.
Ollama Watch analysis
What happened
The guide describes how an agent request typically contains a long input and a short output, often just a tool call. Prefix caching lets the model reuse work from earlier turns. But the cache only matches up to the first change in the prompt: alter something near the beginning and the model may have to process everything after it again.
The authors’ practical advice includes keeping system prompts stable and putting changing details, such as timestamps or working directories, in the task message instead. They also suggest choosing a reasoning-effort setting at the start of a session, trimming oversized tool results while clearly saying what was omitted, and choosing carefully how much conversation history subagents inherit. The guide also discusses routing requests back to the machine holding a session’s cache.
Why it matters
These are the mechanics behind why a coding agent can feel quick at first and increasingly sluggish as a task grows. A changing system prompt or a tool result that quietly cuts off halfway through a file can waste computation, increase cost and leave the model acting on incomplete information.
The advice is aimed at people building agent harnesses and inference systems, not a switch most users can flip in a chat window. But it gives developers concrete places to look when long sessions become slow or behave oddly.
Our read
The most useful message is that prompt structure is part of performance engineering. Keep stable information stable, make omissions explicit, and do not assume a subagent needs the parent’s entire conversational autobiography. The guide comes from people working on both sides of the exchange: Ollama on harness integrations and Together AI on inference. Its examples are practical guidance, not a guarantee of identical gains across every model and setup.
What to watch
- Whether agent harnesses make cache-friendly prompt design easier by default.
- How inference providers expose cache-aware routing and its effects on cost and latency.
- Whether teams make tool-output truncation visible enough to prevent agents acting on missing context.
Discussion spark: Should agent developers prioritise preserving prompt caches, or is the extra complexity worth it only for long-running tasks?
Sources and evidence
- Source update (18 November 2026, 00:00 UTC)
not affiliated with or endorsed by Ollama