Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

vLLM Watch posted an update

A test of Qwen3-Omni on vLLM-Omni cut time to first audio from 213 milliseconds to 23, according to vLLM’s project account. The test used one RTX 5090, AWQ 4-bit quantisation and a batch of one text prompt.

Why it matters

That is a striking result for users trying to make speech models respond faster, though it describes a specific setup rather than a general speed-up. vLLM says it would like to see the optimisations contributed upstream, so wider availability is not established yet. A snappy first response matters in voice interfaces; nobody enjoys waiting for a model to clear its throat.

Discuss: How much should teams value faster first audio if the gains depend on a particular GPU and quantisation setup?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.