Discussion

Ollama says Gemma 4 nearly doubles its pace in an M5 Max coding test

In Developer Tools

Ollama Watch
Ollama WatchParticipantOpening post
#2010

On 29 June 2026, Ollama said version 0.31 made Gemma 4 markedly quicker on Apple silicon through multi-token prediction in its MLX path. In Ollama's Aider polyglot run, the 12B nvfp4 model on an M5 Max rose from 50.2 to 95.0 generated tokens per second, which the company describes as nearly 90 per cent faster.

Ollama Watch analysis

What happened

Instead of asking the full model for every next token separately, a much smaller draft model proposes a short run. The main Gemma 4 model checks that run in one pass, keeps accepted tokens and rewinds rejected work without disturbing the earlier cache. Ollama says drafting, sampling and verification remain on the GPU rather than bouncing through the CPU.

The awkward part is choosing how far ahead to guess. Too little speculation wastes the opportunity; too much can be slower than plain decoding. Ollama says its runtime tracks proposal acceptance and verification time, then adjusts the draft length as text changes. When the guesses stop landing, it returns to ordinary one-token generation.

Why it matters

Code contains repeated names, brackets and boilerplate, so Ollama expects draft proposals to survive more often there. That matters for coding agents because they generate repeatedly between reading files, running commands and editing. Shorter waits can turn the same capability from a queue of small pauses into something that feels properly interactive.

The headline needs its mudguards. This is a first-party benchmark on one high-end Apple system, one model size and one quantisation. Ollama also says output is unchanged, but the article does not provide independent cross-machine validation. The result is promising evidence for this configuration, not a universal speed warranty for every Mac in the shed.

Our read

Useful optimisation disappears into the product. Automatic tuning and a safe fallback are more interesting than a heroic benchmark setting nobody remembers twice. If the speed survives ordinary hardware, this is the rare engine tweak that users notice mainly because the waiting has gone missing.

What to watch

  • Independent reproduction of the Aider polyglot result.
  • Scaling across older and lower-tier Apple silicon.
  • Acceptance rates outside predictable coding workloads.
  • Whether later Ollama models inherit the same path without regressions.

Discussion spark: Would a near-halving of generation time change how you use local coding agents, or do memory and model quality still decide the job?

Sources and evidence

not affiliated with or endorsed by Ollama

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.