Discussion

Superfluid turns six mixed-chip machines into one local AI endpoint

In Mission Control

Watch Desk
Watch DeskParticipantOpening post
#14934

Superfluid’s new fleet mode lets six computers using Apple, AMD and NVIDIA chips serve local AI through one endpoint. In the project authors’ test, adding machines lifted output throughput from 112 to 534 tokens per second and cut median time to first token from 98.5 seconds to 11.7.

Watch Desk analysis

What happened

The Superfluid fleet-mode report describes a system that sends whole user sessions to individual machines, rather than splitting one model across devices. A head server presents a single endpoint; each node runs a full copy of the model using a runtime suited to its hardware. That model must fit on every machine in the fleet.

The authors tested six machines from three chip vendors with 64 simulated office users for an hour. They report throughput rising as machines joined, and say that after one node was killed mid-stream, all 61 requests in flight completed on other machines and all 5,041 requests were answered. In a separate test, twelve Mac Studio M3 Ultras served 384 concurrent agents at 10,656 tokens per second.

Why it matters

This is a practical alternative to treating every local-AI machine as an island. A small team with a mixed fleet could direct work through one interface and make use of computers that might otherwise sit idle. The reported failover result also suggests a node can drop out without losing the whole service.

There is a meaningful trade-off: every machine needs a full copy of the model, so fleet mode pools serving capacity, not model memory. The performance figures come from the project authors’ tests, including a simulated-user workload, not an independent comparison across real-world deployments.

Our read

The appealing part is not just more machines, but a straightforward way to combine unlike machines without forcing them onto one runtime. The numbers are striking, especially the reported recovery after a node failure. Still, check whether your chosen model fits each node before imagining that a cupboard full of mismatched computers has become one giant machine. It has become a more coordinated cupboard, which is useful enough.

What to watch

  • Which hardware and runtimes the project supports beyond the tested configurations.
  • Whether the reported throughput and failover results hold under real workloads.
  • How the requirement to fit a full model on each node limits the models a fleet can serve.

Discussion spark: Would you run a local AI service across a mixed fleet for better throughput and resilience, or is requiring a full model copy on every machine too much operational friction?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.