AWS has published a walkthrough for training a smaller shopping agent from a larger model’s successful tool-use traces. The useful detail is the method: keep the teacher’s solved attempts, train the student on each step, then test it on problems held back from training.
AWS AI Watch analysis
What happened
The example uses GLM-5.3 on Amazon Bedrock as the teacher and Qwen3.5-4B as the student. AWS says its ShoppingBench setup contains 750 problems, split into 426 for training and 324 for evaluation. The teacher tries each training problem twice, producing 850 trajectories; the benchmark’s scorer filters those down to successful runs.
Rather than training on a final answer alone, the walkthrough turns each successful trajectory into step-by-step examples of reasoning, tool calls and returned observations. AWS says the resulting dataset expands to about 2,600 training rows and 130 validation rows. It recommends splitting by problem, not by individual row, to avoid near-duplicate steps leaking across the training and validation sets.
Key findings
- Distil behaviour, not just answers
Each training example teaches the student the next reasoning-and-action step in a multi-step tool-use task. - Let a task scorer filter the teacher’s attempts
The example keeps trajectories that solve the benchmark problem, rather than treating every teacher run as a useful demonstration. - Keep evaluation problems separate
AWS holds out whole problems so similar steps from one task do not appear on both sides of the evaluation.
Why it matters
An agent can make several model calls in one task, so using a frontier model at every step can make a useful application expensive to run. AWS’s approach offers a concrete route to training a smaller model for a bounded workflow, using the teacher’s text outputs rather than access to its weights or internal logits.
The recipe depends on having tasks with a reliable scorer: without one, deciding which teacher runs are worth imitating becomes much harder. The post describes a worked example, not proof that the resulting student matches its teacher in other settings.
Our read
The strongest part is the attention to evaluation hygiene. Splitting by problem is less glamorous than naming a new agent, but it helps avoid a flattering test that has already seen near-copies of its training examples. Teams with a narrow task and a dependable grader have a clear experiment to try; everyone else should start by asking how they will tell a successful trajectory from a convincing failure.
What to watch
- Whether AWS publishes the held-out student-versus-teacher results and evaluation details.
- How much the approach depends on the quality and coverage of the task-specific scorer.
- Whether the recipe transfers beyond the shopping tasks used in the example.
Discussion spark: For a tool-using agent, would you trust a smaller model trained on a frontier model’s successful traces, or do you need the frontier model making the decisions at runtime?
Sources and evidence
- Build efficient agents with knowledge distillation on Amazon SageMaker AI (2 November 2026, 17:01 UTC)
not affiliated with or endorsed by Amazon Web Services (AWS)