Multiverse Computing says it has found a more effective way to remove whole transformer blocks from large language models, using an Ising-style optimisation problem to account for how those blocks interact. The claimed payoff is substantial: at 50% depth compression on Llama-3.3-70B-Instruct, its method reportedly held MMLU near 77 while the strongest comparison method fell into the mid-50s.
Hugging Face Watch analysis
What happened
The work, published in a post on Hugging Face, treats each transformer block as a binary choice: keep it or remove it. Instead of ranking blocks independently, the method models pairwise interactions between removal decisions with a Hessian-derived energy function. That lets the team search for combinations of blocks that should preserve more capability after compression.
Multiverse Computing says the approach can evaluate candidate configurations cheaply after one calibration step, then use brute force or classical and quantum-inspired solvers to find promising low-energy combinations. Its open-source tabu solver reportedly reached the best verified states in seconds for the harder cases, while exhaustive checking of roughly 29 billion configurations took about two days on one GPU.
The post reports results across Llama-3.1-8B-Instruct, Qwen3-14B and Llama-3.3-70B-Instruct, as well as a hybrid NVIDIA Nemotron model mixing attention, Mamba2 and mixture-of-experts layers. It also says the best-performing configuration is not always the lowest-energy, or ‘ground state’, solution. An excited state that removed an early block reportedly beat the ground state after light retraining.
Why it matters
Block removal is an appealing form of compression because a shorter model should need less memory and run faster. The catch is that removing the wrong layers can turn a clever optimisation into an expensive way to make a model worse. This work’s useful idea is to treat the choices as connected rather than pretending every block lives on its own little island.
The headline result is company-reported, not an independent replication, and the gains are strongest under aggressive compression. That is exactly where the technique could matter most for operators trying to squeeze capable models into smaller or cheaper hardware.
Our read
This is a promising compression result with a properly testable centre: compare the retained quality and real inference cost against the baselines, on the same hardware and workloads. The physics framing is memorable, but the practical prize is simpler. If the method survives outside the authors’ benchmark setup, model pruning becomes less like deleting pages at random and more like solving a structured engineering problem.
What to watch
- Independent replications of the reported MMLU and other benchmark gains.
- End-to-end latency, memory and energy results after pruning, not just benchmark scores.
- Whether the method remains effective on newer hybrid and mixture-of-experts models.
- How much calibration data and retraining are needed for production use.
Discussion spark: Should model-pruning methods be judged mainly by benchmark retention, or by the full result on real hardware including latency, memory and energy costs?
Sources and evidence
- Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem (21 September 2026, 13:44 UTC)
not affiliated with or endorsed by Hugging Face