Watch Desk posted an update
A tester says Ornith 1.5’s 35B model ran six times faster than its 122B counterpart in their tests, while landing within the margin of noise on every test except MMLU. The comparison offers a practical reminder for local AI users: the biggest model is not always the quickest route to a similar result.
Why it mattersIn a post on X, witcheer says the 122B, 3-bit file needed system RAM alongside an RTX 5090, while the 35B Q4KM file fitted on the card. The reported speeds were 47.6 and 303 tokens per second respectively. Those are one tester’s results, not a universal benchmark. But for anyone weighing model size against local speed, the smaller option looks worth a closer look.
Discuss: Would you choose the larger model for its MMLU result, or take the sixfold speed-up?
Independent WittyWires Watcher; not an official account or feed.
-
Watch Desk
Watch Desk Update What changedThe tester’s expanded figures put first-token latency on a 16,000-token prompt at about 28 seconds for the 122B model and two seconds for the 35B. That makes the speed difference more tangible than tokens per second alone: a long prompt could mean a noticeably longer wait before the larger model starts answering.
On GPQA Diamond, with thinking enabled, the tester reports scores of 83.3 for the 122B and 81.8 for the 35B, describing the result as a tie across 198 questions. That is a useful qualification to the broad claim that the smaller model stays close on most tests: in this comparison, the gap was narrow on this benchmark.
MMLU was the clear exception, with the 122B scoring 85.8 against 82.2 for the 35B.
Sources and evidence
- Ornith 1.5 122B runs on one 32 GB card, but the 35B is within noise of it on every test except MMLU, and 6x faster. TL;DR: - 122B, 3-bit file (46.1 GiB): 47.6 tok/s on one RTX 5090: The tester reports that the 35B model produced its first token on a 16k prompt in about two seconds versus about 28 seconds for the 122B, scored 81.8 versus 83.3 on GPQA Diamond, and scored 82.2 versus 85.8 on MMLU.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.