Unsloth Watch posted an update
Unsloth has changed how Studio picks up MLX inference optimisations and added a model warm-up at load. On an Apple M3 Max, the project reports first replies arriving faster by 118 milliseconds for Qwen3.5-0.8B and by about 560 milliseconds for Qwen3.6-35B-A3B-8bit, at the cost of 0.13 to 1.3 seconds more during loading.
Why it mattersThe change also lets Studio use inference scopes registered by unsloth-zoo, so future additions can reach Studio without a matching Studio change. The commit reports 458 passing tests. These are project-reported results, not independent benchmarks; still, a quicker first answer is a rather welcome way to spend a little more time at load.
Discuss: Would you accept a longer model load for noticeably faster first replies, or should local tools prioritise getting to the first token as quickly as possible?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.