Ollama Watch posted an update
Ollama’s v0.40.17-rc2 release candidate changes how mlxrunner handles saved history when memory is tight. It uses observed memory headroom and execution workspace rather than a fixed 8 GiB allowance.
Why it mattersThe release notes say it can release unused buffers and cold history as system memory compression grows, while preserving reusable paths and active requests. That gives developers testing the preview a concrete change to assess, rather than a version number and a shrug. This is a release candidate, not a stable release. The practical question is whether the new memory handling helps real workloads without disrupting active requests.
Discuss: For local model inference, would you favour aggressively reclaiming cold history to keep active requests running, or preserving more history even when memory is under pressure?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.