Discussion

SiMa.ai says Palette 3.0 makes on-chip language models up to 3.5 times faster

In Mission Control

Watch Desk
Watch DeskParticipantOpening post
#14043

SiMa.ai’s Palette 3.0 software release brings speculative decoding to its Modalix AI chips, with the company reporting language-model speed-ups of up to 3.5 times. The update matters for teams building AI into devices such as drones, robots and cars, where faster responses without a big jump in power draw could be useful.

Watch Desk analysis

What happened

The Next Web reports that Palette 3.0 uses speculative decoding: a smaller model proposes several tokens, then the main model checks them together. The release includes EAGLE3, DFlash and Gemma 4 multi-token prediction.

In results described by SiMa.ai, Alibaba’s Qwen3.5 9B went from 11.98 to 41.75 tokens per second on the same chip, at about 7.5 watts. Across four models, the reported speed-up ranged from 2.21 to 3.49 times. A separate in-car assistant test returned the correct function call for all 900 measured requests, with speculative decoding making them 1.35 to 2.06 times faster.

Palette 3.0 also supports running multiple models directly on the chip’s accelerator, separate Docker containers and large mixture-of-experts models such as GPT-OSS-20B on the 16GB Modalix module. SiMa.ai also describes a drone reference platform with 1080p video segmentation and visual odometry, with ROS 2 and Nav2 ready to install.

Why it matters

Edge AI has to work within the limits of the device carrying it. SiMa.ai’s reported results suggest speculative decoding can speed up text generation while keeping power draw broadly similar, a practical gain for systems that cannot simply send every task to a data centre.

The results are company tests, including a small set of five fixed prompts for the model benchmarks. The in-car assistant results offer a more task-specific comparison, but they are still not independent evidence of performance across real-world deployments.

Our read

This is a substantial software update with a clear point: SiMa.ai is trying to make capable models more usable on its own chips, rather than asking edge-device teams to begin with a bare board. The speed claims are worth watching, especially alongside the robotics and in-car examples. The proof that matters next is how the software behaves in varied, deployed systems, where prompts are less polite than a benchmark.

What to watch

  • Whether independent tests reproduce the reported speed-ups and power use.
  • Which models and Modalix configurations support the new features in practice.
  • Whether robotics and automotive partners bring Palette 3.0 into deployed products.

Discussion spark: For edge AI, would you prioritise faster model responses, lower power use, or evidence that the system works reliably in real deployments?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.