llama.cpp has released version 0.9.0, with changes spanning model support, hardware backends and inference performance. For users and operators, the practical story is a broad set of additions across the project, including multi-node tensor parallelism and support for several newer model features.
llama.cpp Watch analysis
What happened
The llama.cpp 0.9.0 release notes list changes since v0.8.0. Among them are support for Metis-1.6 models, Nemotron 3.5 Transcribe and Chatterbox TTS, plus updates across CUDA, Vulkan, OpenCL, SYCL, Metal, ROCm/HIP and Hexagon.
Our top picks
- Run tensor-parallel workloads across nodes
The release adds NCCL support for multi-node tensor parallelism. - Add more Gemma 4 optimisations
The notes describe graph optimisations with “massive gains” on prompt processing for E2B and E4B; no benchmark figures are provided here. - Expand Adreno support
An OpenCL change improves q20 GEMV for Adreno, while Hexagon gains several new operations and performance work. - Support more models and media tasks
The release adds Metis-1.6 model support, Nemotron 3.5 Transcribe support and Chatterbox TTS support. - Broaden Vulkan and OpenCL capabilities
Vulkan gains a tensor-parallel all-reduce function; OpenCL adds initial support for IQ1, IQ3, IQ4 and q20. - Improve LFM3 and Gemma 4 handling
Changes include NVFP4 LFM3 GGUF support and resizing Gemma 4 images without letterbox padding.
Why it matters
This is more than a version-number shuffle: the notes describe changes for people running models across different hardware, as well as additions that widen the range of models and media workflows llama.cpp can handle. Multi-node tensor parallelism is especially notable for operators looking to spread inference across machines, rather than keeping the whole job on one box.
Our read
There is plenty here for practitioners to investigate, and the release notes give concrete places to start. The Gemma 4 performance wording is promising, but without figures it is a claim to test, not a benchmark to quote as settled fact. For everyone else, the useful takeaway is that llama.cpp 0.9.0 broadens model and backend support in several directions at once.
What to watch
- Whether independent benchmarks confirm the reported Gemma 4 prompt-processing gains.
- How multi-node tensor parallelism performs in real deployments.
- Which newly supported models and backend changes prove most useful to users.
Discussion spark: For local inference, which matters more to you: support for more models, or the ability to spread workloads across multiple machines?
Sources and evidence
- v0.9.0 (4 November 2026, 13:08 UTC)
not affiliated with or endorsed by llama.cpp or ggml-org