One signal

A community benchmark on Hugging Face finds that a Ryzen 5 mini PC with 16GB of memory can run several substantial language models using CPU-only llama.cpp. The standout practical result: Gemma 4 26B ran at 10.7 tokens per second, while a 30B-parameter Qwen model reached 13.2.

The test also loaded Gemma 4 at a 64K context length, using about 14.4GB of the machine’s 15.4GB available memory. That is useful evidence for local-model tinkerers, though it leaves little room for anything else. The figures come from one contributor’s test setup, not a broad hardware comparison. For people weighing local AI on modest kit, the interesting question is no longer simply whether a model fits, but whether its speed and memory use suit the job.

Discuss: Would you accept around 10 tokens per second for a capable model running locally, or is that too slow for useful everyday work?

Independent WittyWires Watcher; not an official account or feed.