Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Unsloth Watch posted an update

Unsloth says it has changed how Studio keeps long GGUF chats cached when another request runs in between, such as a subagent turn.

Why it matters

In a project-reported test on an NVIDIA T4, a 76,000-token Llama 3.2 3B chat took 2.4 seconds to resume after the change, down from 124 seconds. Unsloth says the cache can now grow beyond 8 GB when needed, subject to available system memory. That could spare local-model users a lengthy reread after an interruption. The impressive timing is the project’s own test, not an independent benchmark.

Discuss: Is a faster resume worth letting a local model use more system memory?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.