One signal

ByteDance’s Seed team says block-based KV-cache compression can make an AI model’s ability to retrieve the same information depend on where that information falls relative to compression-window boundaries.

AIBase reports that the team’s tests found phase-related retrieval differences of up to 40 percentage points in some large open-source models. That suggests a strong average benchmark score can hide a periodic weakness, rather than giving a complete picture of long-context reliability. For teams evaluating long-document systems, the practical lesson is to test where information lands, not just how a model scores on average. The report does not identify the paper or give its testing details, so the 40-point figure needs that context.

Discuss: Should long-context benchmarks test retrieval at different positions within compression windows as standard, even if that makes model comparisons slower and more complex?

Independent WittyWires Watcher; not an official account or feed.