Gradium’s text-to-speech model is now available as a dedicated deployment in Baseten’s Model Library, with a design intended to get voice agents speaking before an AI has finished composing its reply. Baseten says its benchmark recorded a median time to first audio of 155 milliseconds, a useful target for conversation where silence can make even a clever agent feel rather less clever.
Baseten Watch analysis
What happened
Gradium uses Delayed Streams Modeling, a method developed by Kyutai, to generate text and audio as parallel streams. The audio runs a few steps behind the text, so the model can begin speaking after the first few tokens rather than waiting for the whole sentence. The streams advance on a shared 80-millisecond clock.
Baseten says the Coval TTS benchmark recorded a median time to first audio of 155ms, with an interquartile range of 2ms. The benchmark was captured on 4 May 2026. Baseten also says CUDA Graphs cut median latency by more than 2.5 times at small batch sizes, and that its delivery network can make cold starts two to three times faster. Those are figures reported in Baseten’s account, not a promise that every deployment or workload will see the same result.
Why it matters
For a voice agent, the wait before the first spoken word is part of the experience, not a minor technical footnote. Starting speech while the language model is still generating could leave time for it to retrieve information or reason without making the caller sit through a conspicuous pause.
The deployment story matters too. Voice traffic can surge, and a model that starts quickly in a benchmark is less useful if new instances take ages to come online. Baseten says its delivery network speeds those cold starts by mirroring model weights and distributing them through local and shared caches. That links the speech model’s claimed responsiveness to the infrastructure serving it.
Our read
This is a credible, practical angle on voice AI: not simply making speech sound more natural, but coordinating generation and deployment so a conversation can keep its rhythm. The 155ms figure is promising, while the small-batch and cold-start claims deserve testing on the workloads that matter to developers. Nobody wants a voice assistant with the conversational timing of a lift that has stopped between floors.
What to watch
- How the 155ms median holds up across different hardware, voices and production workloads.
- Whether streaming speech remains natural when the text generator changes direction or pauses.
- How Baseten’s cold-start figures translate into response times during real traffic spikes.
Discussion spark: For voice agents, which matters more to making a conversation feel natural: lower time to first audio, or speech that can adapt smoothly when the AI is still deciding what to say?
Sources and evidence
- Source update (13 October 2026, 13:32 UTC)
not affiliated with or endorsed by Baseten