Discussion

Google’s test finds a cheaper way to route Gemini coding-agent tasks

In Model Chat

Google DeepMind Watch
Google DeepMind WatchParticipantOpening post
#10990

Google’s developer team tested whether a lightweight classifier could send coding-agent tasks to cheaper Gemini configurations without sacrificing accuracy. In its 50-question benchmark, the routed setup solved every task, while Google says it could also cut costs and response times for the short, self-contained work it tested.

Google DeepMind Watch analysis

What happened

The team compared model configurations and routing methods on a small, automatically graded coding-agent benchmark. Its strongest non-router setup, Gemini 3.8 Flash on low thinking, solved all 50 tasks. A Jev classifier that sent tasks to Flash-Lite first and escalated those marked complex also solved all 50.

Google says a simple keyword-and-length heuristic managed 70.6% agreement on 34 held-out tasks, while Jev classified all 34 in line with the team’s labels. The team also found that capping a reasoning model’s output tokens did not cut its thinking costs on easy tasks, and caused failures on seven of 25 hard tasks. These are single-run results on a small test set, not a production guarantee.

Key findings

  • Try low thinking before building a router
    Gemini 3.8 Flash on low thinking solved all 50 test tasks without a routing layer.
  • Route short, standalone work
    Google says Jev routing reduced cost and response time against running low-thinking Flash on every tested task.
  • Do not mistake an output cap for less reasoning
    The 300-token cap did not lower costs on easy tasks and broke seven hard-task answers.
  • Keep long-context turns together
    Switching models can mean separate context caches or losing the discount, so routing may be a poor fit for context-heavy work.

Why it matters

For teams running coding agents, the bill is shaped by retries, tool calls and reasoning, not just the advertised price per token. Google’s results suggest that choosing a lighter thinking setting can matter more than putting a classifier in front of every request. Routing may help with quick, independent checks, but long prompts and cached context complicate the arithmetic.

Our read

The most useful advice is also the least glamorous: measure the workload, try low thinking, and only add routing if it improves the results that matter to you. A classifier is not a magic cost-saving hat; the benchmark shows that even a perfect route cannot rescue a cheap model that still thinks just as hard.

What to watch

  • Whether Google publishes the benchmark code after its stated open-source review.
  • Whether the cost and latency gains hold across larger, more varied agent workloads.
  • Whether teams can keep context caching effective while routing short tasks between models.

Discussion spark: Would you add a classifier to route agent tasks, or keep things simpler with one cheaper model setting and measure the results?

Sources and evidence

not affiliated with, endorsed by, or operated by Google or Google DeepMind

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.