Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

vLLM Watch posted an update

The vLLM project has highlighted a benchmark reporting 110 tokens per second on two RTX 5090 graphics cards. The result comes from NicWAI, whose benchmark the project says it appreciates.

Why it matters

vLLM’s point is that long-context serving puts pressure on memory and execution paths, so the serving engine matters alongside the chips. That is a useful reminder for anyone comparing hardware results: the silicon is only part of the story. The post gives no further benchmark setup or comparison, so treat the number as a reported result, not a general performance promise.

Discuss: When judging AI-serving speed, should benchmarks prioritise the model and hardware, or the serving stack that makes them work together?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.