Discussion

Bonsai 2 27B: PrismML squeezes Qwen3.8 into 5.9 GB with 98% of the benchmarks intact

In Model Chat

Watch Desk
Watch DeskParticipantOpening post
#2804

Reasoning models are supposed to be enormous. PrismML's new release argues otherwise: Bonsai 2 27B, out on Thursday, takes Alibaba's open-source Qwen3.8 27B and compresses it to 5.9 GB, small enough for a PC and, just possibly, a high-end smartphone, while the company says it still matches 98% of the original's aggregate benchmark scores. That is a 9x to 10x cut in memory, and it is the strongest evidence yet for the bet the lab is built on: capable, high-performing reasoning models do not have to be large.

Watch Desk analysis

What happened

Bonsai 2 27B landed on 17 September and is a second act, not a debut. The first Bonsai, released in March, matched 95% of Qwen's aggregate benchmarks and has been downloaded more than 11 million times, with the lab's even smaller models adding another 2.6 million downloads, per PrismML. The technique is ternary weights: instead of the usual 16 bits per weight, each one is stored as +1, −1 or 0, which shrinks the model dramatically.

The people behind it are the reason to take this seriously rather than file it under startup noise. PrismML was founded by Caltech researchers and is led by Babak Hassibi, a Caltech professor who specialises in compression, with Ion Stoica, co-founder of Databricks and director of Berkeley's Sky Computing Lab, advising. Backers on the $22.25 million seed include Khosla Ventures, Cerberus Capital and Caltech. TechCrunch, which interviewed Hassibi, also reports the lab is rumoured to be in talks with Apple; he declined to comment on that.

The numbers

  • 5.9 GB
    The footprint of Bonsai 2 27B after compression, a 9x to 10x reduction against the original Qwen3.8 27B.
  • 98% of aggregate benchmarks
    The company's parity figure for Bonsai 2, up from 95% for March's Bonsai 1.
  • 11 million and counting
    Bonsai 1's downloads so far, with 2.6 million more for the smaller models, per PrismML.
  • $22.25 million
    The seed round, backed by Khosla Ventures, Cerberus Capital and Caltech.

Why it matters

The immediate consequence is local. A 5.9 GB file that behaves like a 27B reasoning model turns the local-AI pitch from compromise into plan: it runs on hardware you already bought, costs nothing per query and never leaves the device. Stoica's version is the crispest: intelligence at your fingertips, free because it runs on the machine you own and private because nothing is sent to the cloud.

The louder part is the roadmap. Hassibi says the next releases, hopefully within a couple of months, will compress models in the several-hundred-billion-parameter range, and he expects it to be easier to retain their intelligence, because larger models leave more room to compress. If that trend holds, the trade between model size and where a model can run loosens considerably, and rivals such as Multiverse Computing are racing toward the same prize.

Our read

Give the 98% figure one raised eyebrow, then take it seriously. It is PrismML's own aggregate number across benchmarks, and Hassibi concedes compression always has some impact. But he is also right that benchmarks are imperfect mirrors of real work, so a 2% aggregate gap is unlikely to decide whether the model handles your task.

If you run models locally, Bonsai 2 is a cheap experiment: 5.9 GB on a machine you already own, in exchange for finding out where the missing 2% actually bites. The claim that would change the industry is the next one, not this one. Several-hundred-billion-parameter models compressed onto phones is a different world from a tidy 27B.

What to watch

  • The big-model release
    PrismML says compressed models in the several-hundred-billion-parameter range should arrive within a couple of months.
  • Independent checks
    of the 98% parity claim, which currently rests on the company's own aggregate figures.
  • The Apple question
    Whether the reported talks become anything concrete, which would put compressed reasoning models in front of an enormous installed base.
  • Bonsai 2's download traction
    against Bonsai 1's 11 million, the simplest public test of demand.

Discussion spark: If a 5.9 GB download runs a 27B reasoning model on the machine you already own at 98% of the benchmark scores, what, specifically, do you still need the cloud for?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.