Prefrontal AI and ETH Zurich researchers say their Latent Sparse Attention method can sharply reduce the time AI models spend preparing prompts with up to 10 million tokens. In tests reported by 36Kr, it cut that preparation time for individual attention layers by up to 72.4 times, while keeping accuracy close to existing approaches on several tasks.
Watch Desk analysis
What happened
Latent Sparse Attention, or LSA, changes how a model searches through a very long context. Instead of scoring every key one by one, it groups similar keys into buckets and searches those groups. A hierarchical version builds a tree of buckets, allowing the search to narrow down through levels rather than scan the entire context.
The researchers tested a version adapted to existing models on Qwen3.8-27B and GLM-5.3-Flash. At 10 million tokens, they report 72.4 times faster prefill for one Qwen attention layer than full attention, and 22.7 times faster for one GLM layer than its native sparse attention. In an SGLang deployment, they report time-to-first-token reductions of 9.2 times for Qwen and 2.2 times for GLM at that context length.
The paper also describes two routes to using LSA: distilling it into an already-trained model, or training a model to use it natively. The report says the adapted Qwen model scored 29.1 on the researchers’ PaperTrail benchmark at 10 million tokens, compared with 19.8 for full attention.
Why it matters
Long prompts can mean long waits before a model begins answering. The reported deployment results suggest that improving attention search could make very large contexts more practical, not merely make a single benchmark chart look sprightlier.
The headline speed-ups are for individual layers, not whole models, and the reported end-to-end gains are smaller. The results come from the researchers’ tests as reported by 36Kr, so they are a promising technical result rather than an independent comparison across workloads.
Our read
This is a substantial attempt to tackle a real bottleneck: finding the useful parts of an enormous context without inspecting everything in detail. The 10-million-token results are striking, especially the reported reductions in time-to-first-token. The next test is whether other teams can reproduce the gains on different models and realistic workloads. That is where a clever architecture stops being a clever paper and starts earning its keep.
What to watch
- Whether the paper and code lead to independent reproductions of the speed and accuracy results.
- How LSA performs on other models and real long-context workloads.
- Whether the reported end-to-end gains hold up as models and deployment setups change.
Discussion spark: Would you trust a model to search a huge context through learned groups of relevant passages, or do the accuracy trade-offs make a more exhaustive search worth the wait?
Sources and evidence
- 22x Faster Than DeepSeek-Style Sparse Attention at 10 Million Tokens Level – eu.36kr.com (17 November 2026, 04:33 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.