llama.cpp 0.5.0 broadens local AI support with new models, multimodal fixes and faster backends

llama.cpp 0.5.0 is out with support for three new model families, a substantial ggml refresh and a long list of fixes across local inference, multimodal inputs and serving. The practical change is breadth: more models and messier real-world workloads now have a route through the same open-source stack, without requiring a cloud-shaped detour.

llama.cpp Watch analysis

What happened

The official llama.cpp release adds support for HRM-Text, also identified as DFM Mimir 1B, MiMo-V2.6 and DFlash for HunyuanOCR. It also extends handling for Nemotron models, Qwen4Exp operations and Muse Glimmer, while adding parser fixes for Ling 3.0, DeepSeek V3.2/V4, qwen3-coder and Gemma 4.

The release updates ggml to v0.25.0 and adds backend work including CUDA convolution acceleration, Metal mixture-of-experts and SSMCONV fusion optimisations, plus expanded flash-attention and fused-MoE support. Developers can also bind the server to multiple TCP addresses or Unix sockets, expose image inputs in function-call outputs and set sampling parameters through environment variables.

Multimodal users get sanity checks for image-processing layers and allocation, a fix for tall or wide image handling, better video and cache identification, MCP image attachments in tool blocks and support for WEBM uploads in the user interface. Server changes include fixes for router eviction races and child-process handling.

Why it matters

For people running models locally, compatibility is often the story. A model may be impressive, but it is not especially useful if the runtime cannot load it, parse its messages, handle its images or survive a busy server. llama.cpp 0.5.0 addresses all of those awkward edges at once.

The release also shows local inference becoming less like a single clever binary and more like a platform: model conversion, structured outputs, multimodal inputs, routing, APIs and hardware-specific kernels are all moving together. That is good news for builders, though it does make “just upgrade” sound increasingly like an invitation to read the changelog properly.

Our read

This is a substantial infrastructure release rather than a headline-grabbing model launch. Its value lies in making more models and workflows viable across CPUs, GPUs and specialist backends. The most useful upgrades are the unglamorous ones: parser corrections, server lifecycle fixes, safer multimodal handling and broader model coverage.

Teams already using llama.cpp should test the new release against their model mix and backend targets, especially where they rely on speculative decoding, multimodal inputs, routing or custom parsers. The notes establish what changed, but they do not establish identical performance across every device.

What to watch

• Whether the new model support settles quickly across the available conversion and quantisation paths.
• Independent benchmarks for the CUDA convolution and Metal MoE optimisations.
• Follow-up fixes for the newly expanded multimodal and server features.
• How quickly projects built around the new parsers and image-output support adopt the release.

Discussion spark: Is llama.cpp’s growing compatibility surface now more important than chasing peak benchmark speed, or does that breadth risk making local inference harder to maintain?

not affiliated with or endorsed by llama.cpp or ggml-org