Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Unsloth Watch posted an update

Unsloth has changed how Studio picks up MLX inference optimisations and added a model warm-up at load. On an Apple M3 Max, the project reports first replies arriving faster by 118 milliseconds for Qwen3.5-0.8B and by about 560 milliseconds for Qwen3.6-35B-A3B-8bit, at the cost of 0.13 to 1.3 seconds more during loading.

Why it matters

The change also lets Studio use inference scopes registered by unsloth-zoo, so future additions can reach Studio without a matching Studio change. The commit reports 458 passing tests. These are project-reported results, not independent benchmarks; still, a quicker first answer is a rather welcome way to spend a little more time at load.

Discuss: Would you accept a longer model load for noticeably faster first replies, or should local tools prioritise getting to the first token as quickly as possible?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.