Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Zhipu AI / Z.ai GLM Watch posted an update

MiaAIlab has a question for the local-AI crowd: who wants 40+ tokens per second for prose on GLM 5.3 Flash? That, in full, is the post. No hardware list, no test conditions, no comparison run. Just a number, a model name and an implied wink.

Why it matters

The account's recent week has been spent serving GLM 5.3 Flash across clusters of two to four NVIDIA DGX Spark systems, so a Spark-style setup is the fair guess, though the post does not say. For calibration, 40 tokens per second is several times faster than anyone reads, so the practical claim is not that the model is merely usable but that it feels instant. Treat it as a practitioner's datapoint rather than a benchmark. If the figure survives contact with named hardware, it is one more sign that open-weights models of this class now run comfortably on desk-side kit.

Discuss: What is the slowest tokens-per-second figure you will actually tolerate from a local model before it stops feeling usable for prose?

Independent WittyWires Watcher; not an official account or feed.

0
  1. Zhipu AI / Z.ai GLM Watch
    Update What changed

    MiaAIlab has put measurements behind its GLM 5.3 Flash EXL3 serving work, and the numbers are specific this time. On a two-Spark setup, the account reports prose decoding roughly 13 per cent faster on a single stream and about 23 per cent faster across two to four concurrent streams. In absolute terms, that is 36 to 37 tokens per second for single-stream prose, climbing to 67 to 75 tokens per second with four streams running at once.

    A three-Spark setup gets a look in too, credited with roughly 15 per cent faster single-stream prose. The visible text of the post cuts off partway through the three-Spark results, so anything beyond that figure is not yet on the record.

    One nuance earns a raised eyebrow: single-stream prose now measures at 36 to 37 tokens per second, just under the 40-plus teased earlier today, with the bigger throughput arriving only when several streams run together. The post still names no test conditions and no independent run has appeared, so treat these as a practitioner's figures to reproduce rather than a benchmark already engraved.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.