llama.cpp 0.8.0 adds real-time audio, video input and a much broader model toolkit

llama.cpp has released version 0.8.0, a substantial update to the open-source software for running language models. It adds real-time API support and a spread of model, media and hardware improvements that matter to developers building local and self-hosted AI tools.

llama.cpp Watch analysis

What happened

The release notes cover changes across the server, user interface, model support and hardware backends. The server now supports real-time conversations and transcription through /v1/realtime, including tool calls. The multimodal work adds video input for the LFM3 model family and lets the interface accept OGG audio attachments by converting them to WAV.

There are practical changes for people operating the software, too: conversations can be compacted in the interface, and the server gains a device-memory margin setting for dynamic model unloading. The project also lists support for several LiquidAI models and a CUDA optimisation for mixture-of-experts models with 384 experts.

Why it matters

This is not one new model or a fresh coat of interface paint. The release expands what developers can build with llama.cpp, especially around real-time and multimodal workloads, while improving the controls and hardware support needed to run models across different systems. The release notes also list defensive fixes, including a recursion limit intended to prevent stack overflows and checks against truncated GGUF tensor data.

Our read

The notable change is the breadth: audio, video, real-time interaction and backend work all arrive in the same release. For developers already using llama.cpp, the release notes offer concrete reasons to examine an upgrade. For everyone else, this is a useful sign of how quickly the local-model toolkit is growing beyond text prompts and a terminal window.

What to watch

• How real-time conversation and transcription work in deployments using /v1/realtime.
• Which LFM3 workflows make practical use of the new video input support.
• How the hardware and model additions perform across the systems they target.

Discussion spark: Which matters more for local AI tools now: adding real-time audio and video, or making existing models easier to run reliably across different hardware?

not affiliated with or endorsed by llama.cpp or ggml-org