llama.cpp has released v1.0.0, a substantial update to the open-source toolkit for running language models across a wide range of hardware. Among the practical changes are FP8 key-value caching, support for additional multimodal inputs and a more capable server interface, alongside security and reliability fixes.
llama.cpp Watch analysis
What happened
The project’s v1.0.0 release notes collect changes since v0.9.0. They include FP8 KV-cache support, which adds another lower-precision option for managing model context; support for audio input, text-to-speech and additional vision models; and an OpenAI-compatible audio-speech endpoint. The server can also convert JSON-schema text formats for completions and cancel MCP tool calls when a client disconnects or times out.
The release notes also list backend work across CUDA, Metal, Vulkan, ROCm/HIP, SYCL, OpenCL and other platforms. Examples include CUDA support for OCP FP8, a Vulkan Intel FlashAttention kernel, and CPU optimisations for ARM NEON. The changelog names fixes for server-side request forgery in the CORS proxy and media URLs, as well as protections against DNS rebinding and unintended access to child instances.
Why it matters
llama.cpp’s appeal is that local and self-hosted inference is not confined to one accelerator or model format. This release adds useful options at several layers: serving, multimodal input, caching and hardware support. Operators can evaluate the new features against their own models and machines rather than waiting for a single vendor’s preferred stack to catch up.
The release notes document changes, not comparative benchmarks. They do not establish that every model will run faster or use less memory, and the security fixes are not, by themselves, evidence of a reported breach.
Our read
This is a meaningful milestone for a widely used local-inference project, with concrete improvements beyond the round-number version label. FP8 caching and the expanded audio and server capabilities are the standouts; the broad backend work matters because hardware variety is part of the point. If you run llama.cpp, check the release notes for the features and fixes relevant to your setup before upgrading production systems.
What to watch
- Which models and hardware combinations benefit most from FP8 KV caching.
- How the new audio and multimodal paths perform in real deployments.
- Whether the server security fixes prompt follow-up guidance or configuration changes.
- How the backend improvements translate into performance on supported devices.
Discussion spark: For local inference, which matters more in practice: lower-precision caching, wider hardware support, or better server and multimodal features?
Sources and evidence
- v1.0.0 (13 November 2026, 10:20 UTC)
not affiliated with or endorsed by llama.cpp or ggml-org