Intel Neural Compressor 3.10 adds mixed quantisation options and model-free PyTorch quantisation

Intel Neural Compressor 3.10 adds new quantisation options for PyTorch and Keras/JAX, alongside support for more models and hardware. For teams trying to fit AI workloads to available accelerators, the release offers practical new knobs, not just a version number and a ceremonial cake.

Intel AI Watch analysis

What happened

Intel’s release notes date version 3.10 to 29 September. They list experimental NVFP4 quantisation with E5M3 scaling for PyTorch, model-free quantisation for PyTorch, and selective and mixed quantisation for Keras/JAX. The release also includes a vLLM QDQ plugin for accuracy benchmarking through simulation, plus examples for SDXL and Kimi K2.6.

The notes list Intel Gaudi 2 and 3, fourth- to sixth-generation Xeon Scalable processors, and Intel Arc B-series graphics among validated hardware. They also recommend version 3.10 or later to mitigate code CVEs.

Our top picks

• Model-free quantisation for PyTorch
  Adds a route to quantisation that does not require a model-specific setup.
• Selective and mixed Keras/JAX quantisation
  Teams can leave some model components unchanged while choosing different methods and settings for others.
• Experimental NVFP4 with E5M3 scaling
  The new PyTorch support gives developers another low-precision option to test.
• Dynamic fused MoE bias support on Gaudi
  The release adds support for GPT-OSS deployments using Intel Gaudi accelerators.
• A vLLM QDQ benchmarking plugin
  Provides an accuracy-benchmarking option based on simulation, useful when evaluating quantisation choices.

Why it matters

Quantisation can reduce the memory and compute demands of AI models, but a method that works for one framework or model may not fit another. Selective settings matter because they let teams tune parts of a model without applying the same treatment everywhere. The release notes do not give performance results for these additions, so the value will depend on testing with real workloads and hardware.

Intel says version 3.10 or later is recommended to mitigate code CVEs. That is a practical reason for existing users to check their installed version, rather than treating this as a release only for people shopping for quantisation features.

Our read

This is a substantial toolkit update for developers and operators working across PyTorch, Keras and JAX. Start with the feature that matches your workload, check the validated configurations, and measure accuracy as well as speed. “Lower precision” is a useful engineering option, not a free lunch served in smaller numbers.

What to watch

• Whether the experimental NVFP4 support moves beyond its current status.
• How the new quantisation options perform on models and hardware not listed in the release notes.
• Which code CVEs the version recommendation addresses and whether follow-up fixes are needed.

Discussion spark: When choosing a quantisation method, should teams prioritise lower hardware demands or preserving model accuracy, even if that means supporting more than one setup?

Independent WittyWires tracker for public updates about Intel AI. Not affiliated with or endorsed by Intel AI; this is not an official account.