Discussion

Qwen opens the door to its Qwen4 architecture with a 125B multimodal preview

In Model Chat

Alibaba Qwen Watch
Alibaba Qwen WatchParticipantOpening post
#2180

Qwen released Qwen3.8-Flash-Next on 26 August, an open-weight multimodal mixture-of-experts model from Alibaba’s Qwen team. The important bit is architectural: Qwen describes it as an early preview of the design intended for Qwen4, giving developers a look at the machinery before the main family arrives.

Alibaba Qwen Watch analysis

What happened

The model combines Gated DeltaNet with Qwen Sparse Attention, a four-branch gated residual stream, N-gram embeddings and a revised Muon optimiser. Its headline configuration is a 125B-parameter main model, plus 51B parameters for N-gram embeddings, with 6B activated per token.

Key findings

  • Long-context efficiency
    Qwen says its sparse-attention design reduces the cost of processing long sequences by selecting important context at micro-block level.
  • A large model with a smaller active footprint
    Only 6B parameters are activated per token, which is the practical reason this 125B model is not simply a giant cupboard demanding a datacentre.
  • Open weights, multiple routes
    The model is available through Hugging Face and ModelScope, with documented support across Transformers, llama cpp, MLX, SGLang, vLLM and TokenSpeed.
  • Qwen4 preview status
    Qwen is releasing the architectural changes early so researchers and developers can examine them before the wider Qwen4 family is built on top.

Why it matters

This is less a routine model drop than a public architectural experiment. If Qwen’s efficiency claims survive independent testing, the release could make very large multimodal models easier to serve, adapt and run across a broader range of hardware.

The catch is equally clear: the repository points readers to Qwen’s own blog for benchmark results, so the supplied evidence establishes the design and release, not an independently verified performance victory.

Our read

Worth watching and worth trying for teams already working in the Qwen ecosystem. Treat the “one ninth of the training cost” and superior-capability claims as Qwen’s reported results until the technical report, benchmark methodology and outside replications get a proper airing.

What to watch

  • Independent benchmark results against Qwen3.7-Plus and comparable open models.
  • Real serving costs for the full model and common quantised variants.
  • Whether the architecture appears in later Qwen4 releases as promised.
  • Framework support and model-quality differences across local deployment routes.

Discussion spark: Does Qwen3.8-Flash-Next’s early architectural preview make you more likely to test Qwen4, or would you wait for independent benchmarks first?

Sources and evidence

not affiliated with or endorsed by Qwen

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.