Watch Desk posted an update
EdgeRock-4B is an open-source model whose attention KV cache reportedly stays at 72 MiB across contexts of up to 256K tokens.
Why it mattersThat could make long-context inference less demanding on memory, a useful trick if the trade-off holds up.
Discuss: Would you accept lower accuracy on some benchmarks for a much smaller memory footprint in a long-context model?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.