Building AI at the edge is running into a basic problem: the hardware may be available, but sites can struggle to power and cool it. SiliconANGLE’s report from SC26 describes rack densities moving from around 10 kilowatts towards a megawatt, a jump that changes what operators must plan for.
Watch Desk analysis
What happened
Jack Hogan, SHI International’s vice-president of advanced growth technologies, told SiliconANGLE that customers can procure GPUs but may not be able to power or cool them. The report says power, cooling and network throughput are increasingly shaping whether AI systems can be put to work.
Hogan also described why some workloads need to run near their source. Video input, for example, can be difficult to send elsewhere for processing. SHI says it assesses customer goals first, then works back to the right model and hardware, which could include a small language model on an edge device rather than a frontier model in a data centre.
Why it matters
The practical constraint is not just how many accelerators an organisation can buy. A site needs enough electricity, cooling and network capacity to keep them useful. For edge inference, moving data back to a central facility may not be a workable answer either.
That makes infrastructure planning and model choice part of the same decision. A smaller model that can run where the data is generated may be more useful than a larger one that cannot be supported there. The report reflects an industry interview, not a measured survey of all deployments, but it puts concrete limits on the idea that adding GPUs alone settles the problem.
Our read
This is a useful reminder that AI infrastructure is physical, and the unglamorous bits get a vote. Operators planning edge systems should work backwards from the task, location and available power before treating accelerator count as the whole plan.
What to watch
- Whether deployments share measured power and cooling requirements for edge inference.
- How often sensitive or bandwidth-heavy workloads are processed at the source.
- Whether smaller models and alternative processors can meet real deployment needs.
Discussion spark: When an edge-AI site cannot support a large model, should operators prioritise smaller models on-site or send the workload back to a central data centre?
Sources and evidence
- Edge inference hits power and cooling limits, not GPU supply – SiliconANGLE (20 November 2026, 12:55 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.