AWS shows how to build observable, agentic retrieval on Bedrock

AWS has published a hands-on guide to building an agent that can search across two knowledge bases, retrieve repeatedly and return cited answers. Its useful twist is that the tutorial treats observability and evaluation as part of the build, not chores for later.

AWS AI Watch analysis

What happened

The AWS blog post walks through a Python CDK application using Managed Amazon Bedrock Knowledge Bases, Bedrock AgentCore and CloudWatch. The example routes questions between financial and weather knowledge bases, exposes retrieval through an AgentCore Gateway, and uses AgenticRetrieveStream to retrieve iteratively and produce grounded, cited answers.

The guide also provisions two CloudWatch dashboards spanning seven layers of telemetry, alongside on-demand and continuous evaluation. AWS says the managed knowledge bases handle ingestion, storage, indexing and retrieval, so this design does not require a customer-managed vector database.

Why it matters

Agentic retrieval can choose which source to search and go back for more when one pass is not enough. That gives it a practical advantage for questions that cross collections, but it also makes the path to an answer less predictable than a single retrieval call. Seeing what the agent did, and checking answer quality, becomes part of operating the system.

The guide offers readers a concrete architecture to study: two separate corpora, a tool-enabled agent, cited results and monitoring wired into the deployment. It also makes a trade-off clear. Managed storage reduces infrastructure work, while a customer-managed knowledge base remains an option for teams that need to operate their own vector store.

Our read

This is a useful blueprint, not evidence that every agent needs four CDK stacks and a dashboard collection worthy of an air-traffic tower. The strongest lesson is simpler: if an agent can take several retrieval steps, instrument those steps and evaluate its answers from the outset. AWS provides the implementation details; teams should still test the pattern against their own data and operating needs.

What to watch

• Whether AWS adds further managed retrieval and evaluation capabilities to this architecture.
• How teams balance managed knowledge bases against the control of running their own vector store.
• Whether evaluation and telemetry catch retrieval or answer-quality problems in real deployments.

Discussion spark: For production agents, would you choose managed retrieval with built-in observability, or keep control of the vector store and assemble the monitoring yourself?

not affiliated with or endorsed by Amazon Web Services (AWS)