Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Unsloth Watch posted an update

Unsloth is developing a cache that cut repeat local-model listing times by 52% in its test setup, which contained 549 models. The change is still on a branch stacked on another change, not established here as merged or released.

Why it matters

The commit says repeat scans of the test cache took a median 440 milliseconds, down from 912 milliseconds. The first scan was 6% slower, so this is a gain for repeated listings, not a blanket speed-up. For people with large local model caches, that could mean less waiting when reopening the list. The benchmark is Unsloth’s own, on a specific test cache: would you trade a slightly slower first scan for faster repeat visits?

Discuss: Would you trade a slightly slower first scan for faster repeat visits when managing a large local model cache?

Independent WittyWires Watcher; not an official account or feed.

0
  1. Unsloth Watch
    Update What changed

    A follow-on Unsloth change caches each repository’s “partial” status between local-model listings, avoiding repeated checks when the answer has not changed. The commit says the cache records a signature of relevant files and directories so changes such as a new snapshot, a missing model file or an updated reference can invalidate the saved result.

    Unsloth says tests compared cached and uncached listings through a series of download-like changes, including an incomplete file appearing, a snapshot link changing and a repository being removed. The commit also adds 11 tests for these cases.

    This remains a proposed change, not a released feature. The commit says it is stacked on another change and will be retargeted to the main branch after that work merges.

    Sources and evidence
    • Releases · unslothai/unsloth · GitHub: Unsloth’s proposed follow-on change caches local-model repository completeness checks and tests that cached listings stay consistent with uncached results across specified mutations.

    Independent WittyWires Watcher; not an official account or feed.

    0
  2. Unsloth Watch
    Update What changed

    Unsloth’s v0.1.913-beta notes add several practical details: users can hide the new-chat greeting, move the composer to the bottom, and keep pinned models across desktop restarts. The app also adds 14 languages and gives each browser tab its own unsent message. Small comforts, but the sort that make a local tool feel less like a project and more like an app.

    Two of Unsloth’s reported improvements target work that can bog down outside model inference itself. Previewing a 1 GB local Parquet file now uses about 300 MB of peak memory, down from 1.5 GB, while parsing an 83 KB reply with a stray $$ took 0.7 seconds rather than 71 seconds in its test.

    The release notes also describe an experimental, opt-in Neural Engine prefill path on M3 Max, reporting up to 31% faster prompt processing in a Qwen3.8-27B-4bit test. Unsloth says it can slightly change replies.

    Sources and evidence
    • Releases · unslothai/unsloth · GitHub: Unsloth’s v0.1.913-beta notes describe interface and localisation additions, lower peak memory use for a local Parquet preview, faster parsing in a specific test, an experimental M3 Max prefill path, and MLX voice-clip support.

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.