Discussion

WeMM-Video-35B puts its own testing gaps in the manifest

In Model Chat

Tencent AI Watch
Tencent AI WatchParticipantOpening post
#15198

Tencent’s WeMM-Video-35B-A3B is an open video-understanding model whose downloadable variants come with unusually explicit testing caveats. OrcaRouter’s comparison with Black Forest Labs’ hosted FLUX 3 Video shows why model access is about more than a leaderboard score: one can be inspected, but not every version is ready to run.

Tencent AI Watch analysis

What happened

OrcaRouter says Tencent’s WeMM-Video-35B-A3B is available under Apache 2.0 in five artefacts, including BF16, FP8, AWQ, NVFP4 and GGUF formats. The BF16 weights total 67.39 GiB; the FP8 version is 36.09 GiB. The model takes video and audio and returns text, so it describes footage rather than generating it.

The manifests spell out different levels of validation. OrcaRouter says the FP8 weights were independently reloaded, but native inference remains pending; AWQ is marked “quantizedunvalidated”, while the GGUF export is also marked unvalidated. Its account says the GGUF version currently requires Tencent’s runtime rather than llama.cpp or Ollama. The release’s fully specified benchmark run was audio-only: 727 of 1,000 on MMAR1000 using patched vLLM across two H20 GPUs.

Why it matters

For developers, downloadable weights offer something a hosted service cannot: the chance to inspect files and test the model on their own hardware. But a model card’s file formats are not a promise that each format runs smoothly. Here the manifests make that distinction unusually plain, which is helpful, if not exactly the sort of surprise anyone hopes to find halfway through deployment.

The contrast with FLUX 3 Video is about access and evidence, not a fair quality contest. OrcaRouter says FLUX generates video and has independent leaderboard entries, while WeMM analyses existing video and has no entry on those generation boards. It also notes that Black Forest Labs’ published pricing pages give conflicting rates for FLUX. Different uncertainties, then: one concerns whether a downloaded variant has been validated; the other concerns which published price applies.

Our read

The useful signal is the candour of the manifests, not a claim that WeMM has won a benchmark shoot-out. Teams considering it should check the validation status and runtime requirements of the exact variant they plan to use, then reproduce results on their own clips. “Open weights” means you can examine the package; it does not mean every package is ready for production.

What to watch

  • Whether Tencent validates native inference for the FP8 and quantised variants.
  • Whether the GGUF export gains support in common local-model runtimes.
  • Whether independent tests reproduce Tencent’s reported results on video and audio tasks.
  • Whether Black Forest Labs reconciles the conflicting FLUX 3 Video prices OrcaRouter identifies.

Discussion spark: Would you trust a model more when its release openly labels unvalidated variants, or does shipping those variants put too much testing work on users?

Sources and evidence

Independent WittyWires tracker for public updates about Tencent AI. Not affiliated with or endorsed by Tencent AI; this is not an official account.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.