AWS has published a deployable sample that routes coding-agent requests between models on Amazon Bedrock, using recent tool results to guide the choice. Its key idea is to judge routing by the cost of tasks that actually pass their tests, not by token prices alone.
AWS AI Watch analysis
What happened
The sample combines NVIDIA’s open-source NeMo Switchyard with LiteLLM, deployed on AWS using CDK. In its Stage strategy, the router examines the three most recent tool results and chooses between a capable model and a lower-cost one. If the signals are inconclusive, the configuration favours the efficient model. The sample uses Claude Opus 5.5 and NVIDIA Nemotron 3 Super 120B as its candidates.
Developers can compare routed runs with single-model baselines by dividing total run costs by the number of runs whose tests pass. A separate Random strategy supports weighted traffic splits for A/B comparisons. The guide also describes a private-subnet deployment, with a gateway key, request filtering and an allowlist of API paths.
Why it matters
A cheap model call is not necessarily a cheap coding task. If a model needs more turns, retries or time to reach a working result, the apparent saving can vanish. Measuring cost per successful task gives teams a more useful basis for deciding whether routing helps their workload.
It also makes the operational trade-off visible: this is a sample to test and adapt, not a ready-made production service. AWS describes the Switchyard integration as experimental and recommends non-production deployment and review of security and compliance requirements.
Our read
This is a worthwhile recipe because it connects model selection to an outcome developers can check: passing tests. The sample offers a practical starting point, not proof that routing will save money everywhere. Run it against your own tasks and compare the results; the spreadsheet may be less glamorous than the model names, but it is doing the useful work.
What to watch
- Whether the experimental LiteLLM integration and its configuration remain stable.
- How routed runs compare with single-model baselines on real workloads.
- Whether teams can reproduce savings without sacrificing task completion.
Discussion spark: Would you trust a coding-agent router to choose a cheaper model from tool-result signals, or should teams require workload-specific evidence of passing tests before adopting it?
Sources and evidence
- Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard (13 November 2026, 22:30 UTC)
not affiliated with or endorsed by Amazon Web Services (AWS)