AI infrastructure is becoming a systems problem, not just a chip race

AI data centres are increasingly constrained by everything around the processors: memory, networking, storage and power. A SiliconANGLE report from a semiconductor industry summit lays out how those bottlenecks are shaping the next stage of the AI build-out.

Watch Desk analysis

What happened

The report gathers 11 perspectives from executives at the CASPA CEO Summit in California. Western Digital chief executive Irving Tan describes AI as a data system that depends on storage and the wider hardware ecosystem, not compute alone. Other speakers point to optical links, memory pooling, power stability and workload-specific systems as pressing engineering challenges.

Several examples put those constraints in practical terms. Semtech’s Hong Hou says a failing network link can force a job to restart, affecting processor uptime. ScaleFlux’s Greg Ernst says CXL can let servers share memory, while Normal Computing’s Faris Sbahi describes doing some inference calculations directly in memory to reduce cost and energy use. These are attributed views from company executives, not independent comparisons of competing products.

Key findings

• Networks can limit processor uptime
  Semtech’s Hong Hou says link failures can restart jobs, making connectivity a factor in data-centre utilisation.
• Inference is changing how systems are designed
  Rebellions’ Sunghyun Park argues that performance per dollar per watt matters more as agents use tokens continuously.
• Memory pooling is back in focus
  ScaleFlux’s Greg Ernst says CXL can let multiple servers share memory, including older DDR4 modules.
• Power infrastructure moves more slowly than AI hardware
  Schneider Electric’s Melton Chang highlights the mismatch between infrastructure cycles of six to ten years and hardware that turns over roughly annually.
• Optics may help connect larger clusters
  Lumentum’s Michael Hurlston says data-centre operators are looking at optical links as power limits push them towards connecting smaller sites.
• Chip design is using AI agents too
  Siemens Digital Industries Software’s Ankur Gupta says the company uses agents to automate design setup and debugging, alongside GPU-based design engines.

Why it matters

The next gains in AI capacity may depend as much on moving data, supplying power and keeping memory available as on producing faster processors. That shifts attention from the headline chip to the whole system, where one bottleneck can leave expensive compute waiting around.

The report also contains company forecasts and strategic claims, including projections for storage demand and ambitious plans for new computing systems. They are useful signals of what industry leaders are prioritising, not proof that every proposed approach will deliver.

Our read

The chip race has not ended; it has acquired a supporting cast of cables, memory, storage and power systems, all demanding their own starring roles. For anyone tracking AI infrastructure, the useful question is now where the next constraint sits, not simply which processor is fastest.

What to watch

• Whether optical connections and memory pooling move from industry talking points into wider deployments.
• How operators manage power stability as AI hardware changes faster than data-centre infrastructure.
• Which system-level improvements deliver measurable gains in cost, energy use or processor uptime.

Discussion spark: As AI data centres scale, which constraint deserves the most attention: compute, memory, networking, storage or power?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.