Superfluid’s new fleet mode lets six computers using Apple, AMD and NVIDIA chips serve local AI through one endpoint. In the project authors’ test, adding machines lifted output throughput from 112 to 534 tokens per second and cut median time to first token from 98.5 seconds to 11.7.
Watch Desk analysis
What happened
The Superfluid fleet-mode report describes a system that sends whole user sessions to individual machines, rather than splitting one model across devices. A head server presents a single endpoint; each node runs a full copy of the model using a runtime suited to its hardware. That model must fit on every machine in the fleet.
The authors tested six machines from three chip vendors with 64 simulated office users for an hour. They report throughput rising as machines joined, and say that after one node was killed mid-stream, all 61 requests in flight completed on other machines and all 5,041 requests were answered. In a separate test, twelve Mac Studio M3 Ultras served 384 concurrent agents at 10,656 tokens per second.
Why it matters
This is a practical alternative to treating every local-AI machine as an island. A small team with a mixed fleet could direct work through one interface and make use of computers that might otherwise sit idle. The reported failover result also suggests a node can drop out without losing the whole service.
There is a meaningful trade-off: every machine needs a full copy of the model, so fleet mode pools serving capacity, not model memory. The performance figures come from the project authors’ tests, including a simulated-user workload, not an independent comparison across real-world deployments.
Our read
The appealing part is not just more machines, but a straightforward way to combine unlike machines without forcing them onto one runtime. The numbers are striking, especially the reported recovery after a node failure. Still, check whether your chosen model fits each node before imagining that a cupboard full of mismatched computers has become one giant machine. It has become a more coordinated cupboard, which is useful enough.
What to watch
- Which hardware and runtimes the project supports beyond the tested configurations.
- Whether the reported throughput and failover results hold under real workloads.
- How the requirement to fit a full model on each node limits the models a fleet can serve.
Discussion spark: Would you run a local AI service across a mixed fleet for better throughput and resilience, or is requiring a full model copy on every machine too much operational friction?
Sources and evidence
- Distributed Inference for Local AI via Superfluid (20 November 2026, 01:20 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.