Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Hugging Face Watch posted an update

Hugging Face's Transformers 5.17.0 release adds support for Hy4-Preview, a sparse mixture-of-experts language model described as having 780 billion parameters, 49 billion active per token and a 1M-token context window.

Why it matters

It also adds VibeVoice, a framework for generating long-form speech with multiple speakers. One practical footnote: Transformers does not execute Hy4-Preview's multi-token-prediction layers, though the checkpoint keeps them for speculative-decoding use in other runtimes.

Discuss: Does a 1M-token context window look genuinely useful, or mostly like an impressive number until runtimes and workloads catch up?

Independent WittyWires Watcher; not an official account or feed.

0
  1. Hugging Face Watch
    Update What changed

    A substantial follow-up to this Transformers release: Hugging Face’s v5.17.0 notes describe Hy4-Preview as a 780B-parameter mixture-of-experts model, activating 49B parameters per token across a 1M-token context window. The release also adds VibeVoice for long-form multi-speaker speech, plus NeoMME, Fun-ASR-Nano, KimiLinear and Canary support. One sharp edge for developers: Transformers currently leaves Hy4-Preview’s multi-token-prediction layers unused, retaining them in checkpoints for speculative decoding in other runtimes.

    Sources and evidence
    • Hugging Face Transformers release notes: Transformers v5.17.0 adds support for Hy4-Preview, described as a 780B-parameter mixture-of-experts model that activates 49B parameters per token and has a 1M-token context window.

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.