PrismML’s Bonsai 27B compresses a Qwen3.6-27B model’s weights to about 3.9GB, putting a model of this size within reach of more local-AI setups. But that figure is not the whole memory bill: MakeUseOf says you will want a 6GB or 8GB GPU to run it comfortably, and the compression comes with trade-offs in some tasks.
PrismML Watch analysis
What happened
Bonsai is not a new model architecture. MakeUseOf reports that it uses the Qwen3.6-27B model with an unusually compact format: each weight is represented by a sign bit, with groups of 128 weights sharing a scale factor. The report says the weights take about 3.79GB, down from roughly 54GB at 16-bit precision.
The format, Q10, is supported in mainline llama.cpp on CPU, Metal, CUDA and Vulkan, according to the article. The article also says PrismML’s own fork was faster in its CPU prompt-processing comparison, at 13.19 tokens per second versus 7.91 for mainline llama.cpp.
Key findings
- The weights fit in less than 4GB
The model’s weights are about 3.79GB, but that does not include the memory needed to run it. - Context length adds to memory use
PrismML’s figures put peak use at 5.2GB with a 4,000-token context and 11.6GB at 100,000 tokens with an FP16 cache. Four-bit cache quantisation cuts the 100,000-token figure to 6.86GB. - The quality trade-off varies by task
PrismML reports an average score of 76.11 across 15 benchmarks, against 85.07 for the full-precision model. Its reported maths score holds up better than instruction-following and agentic tool-calling scores. - A 4GB GPU may need to share the work
MakeUseOf’s test on a 4GB GTX 1650 Ti could not load the model with full GPU offload at a 4,096-token context. It worked with about half the model offloaded, with the rest running on the CPU.
Why it matters
The attraction is practical: a 27B-class model that would ordinarily need far more memory can now be tried on hardware that many people already own. But “the weights fit” is not the same as “the model runs well”. Context, runtime buffers and the KV cache all need room too, and performance can change sharply depending on whether the GPU or CPU does the work.
The quality figures also deserve a task-by-task reading. PrismML’s reported benchmark average retains 89.5 per cent of the full-precision score, but the article says instruction-following and tool-calling scores fall more than maths. That makes the model more intriguing for some local experiments than a drop-in choice for every agent workflow.
Our read
This is a useful advance in local model compression, not a magic 27B model for any graphics card with 4GB to spare. MakeUseOf reports that an independent study confirmed the released weights are real and work, while noting that the study did not explain PrismML’s method. For now, treat the published performance numbers as PrismML’s claims and the hardware test as one reviewer’s experience.
If you have 6GB or 8GB of GPU memory, Bonsai may be worth testing against the smaller or less-compressed model you already use. If you have 4GB, expect partial offload and slower results rather than a pocket-sized frontier lab.
What to watch
- Whether PrismML explains how its one-bit weight format is produced.
- How Bonsai performs in independent tests across different GPUs and real workloads.
- Whether updates to llama.cpp and other local tools improve speed or reduce memory use.
Discussion spark: Would you accept weaker instruction-following and tool use to run a 27B model locally, or is reliable performance worth using a smaller model?
Sources and evidence
- Source update (5 November 2026, 16:45 UTC)
Independent WittyWires tracker for public updates about PrismML. Not affiliated with or endorsed by PrismML; this is not an official account.