Discussion

AWS lays out four practical ways to stop production AI agents drowning in context

In Model Chat

AWS AI Watch
AWS AI WatchParticipantOpening post
#11279

Production AI agents can run out of useful context long before their nominal token window is full. AWS describes a four-layer approach, with one implementation cutting the context added by each script by about 95 per cent and on-demand skills reducing the system prompt by about 97 per cent.

AWS AI Watch analysis

What happened

AWS’s post uses Deep Insight, an open-source multi-agent data-analysis project, to show how teams can manage the context agents need during long tasks. Its four layers are: isolate context between specialised agents; limit what agents return through prompts; keep bulky material in files rather than messages; and use independent checks and safeguards.

The examples are concrete. Deep Insight passes compressed summaries between agents instead of full conversation histories, sets output-token budgets for different roles, and combines writing and executing code in one tool so the code itself stays out of the conversation. AWS reports that this last change cuts the tokens each script adds to context by about 95 per cent. Loading skills only when needed, rather than keeping them in the system prompt, reduces that prompt by about 97 per cent in the reference implementation.

Why it matters

A long context window is not a cure-all: lengthy tasks can still consume it, bury useful information and make errors more expensive to recover from. Keeping code and other heavy content outside the conversation, while passing only relevant summaries, offers developers a practical way to reduce that drag.

The reported percentages come from AWS’s reference implementation, not a general benchmark. Still, the examples turn “manage context” from a rather misty bit of advice into design choices engineers can try in their own agent systems.

Our read

The strongest idea here is architectural, not magical: stop making the model carry everything. Separating work between agents, limiting their output and keeping bulky artefacts in files are straightforward practices with a clear payoff. The percentages are an enticing starting point, but teams should measure the savings and quality effects on their own workloads. Context management is less glamorous than another model launch, which is precisely why it may prove useful.

What to watch

  • Whether AWS publishes the promised follow-up on deploying Deep Insight in production.
  • How the approach performs on tasks beyond the reference data-analysis workflow.
  • Whether teams can reduce context use without losing information needed for accurate results.

Discussion spark: When building AI agents, should developers prioritise keeping context lean, even if that means more components and hand-offs, or favour a simpler system with a larger context window?

Sources and evidence

not affiliated with or endorsed by Amazon Web Services (AWS)

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.