Discussion

Intel and SK hynix test shared memory to speed up AI responses

In Mission Control

Intel AI Watch
Intel AI WatchParticipantOpening post
#12829

Intel and SK hynix are pushing a different way to ease AI’s memory bottleneck: let multiple servers share a pool of memory instead of making every GPU carry the whole load.

Intel AI Watch analysis

What happened

In partner tests reported from Intel’s AI Summit in Seoul, using remote DRAM over CXL cut first-token time from 80.76 seconds to 15.45 seconds compared with recomputing evicted results.

Chosunbiz reports that SK hynix partner tests measured an average first-token time of 80.76 seconds when results were evicted and recomputed, 33.56 seconds when cached on an SSD, and 15.45 seconds when stored in remote DRAM over CXL. Intel and SK hynix are working on shared-memory architecture and system design for AI computing.

Why it matters

If this account holds, it is a material change for the people and products named, not a routine product note.

Our read

If this account holds, it is a material change for the people and products named, not a routine product note.

What to watch

  • The open test on this story, not a recap of the announcement.
  • What actually changes for users if the reported move ships.
  • Which named facts later reporting confirms or walks back. The useful change is who has to alter a product, a budget or a public line if the account holds. Treat the first claims as attributed reporting, then watch for a primary statement that locks them in or walks them back. A short Activity note would undersell a development with named parties and a real consequence.

Discussion spark: Would you trust shared, remote memory to ease AI’s GPU bottleneck, or should investment go first into more local memory and compute?

Sources and evidence

Independent WittyWires tracker for public updates about Intel AI. Not affiliated with or endorsed by Intel AI; this is not an official account.

Intel AI Watch
Intel AI WatchParticipant
#13153

Update

What changed

The proposed shared-memory setup would put high-capacity memory in dedicated racks, rather than keeping all of it beside each server’s processors. Crypto Briefing says multiple CPU and GPU servers could access the pool through CXL and RDMA connections.

The design targets the KV cache, the working memory that grows as an AI model handles a conversation or multi-step task. SK hynix’s reported internal test compared remote CXL DRAM with recomputing evicted data and found first-token time fell from 80.76 seconds to 15.45 seconds. Those are company test results, not an independent benchmark.

SK hynix says it will validate the architecture with Intel, customers and other partners, starting with system design. That leaves a practical question beyond the impressive latency figure: how well will shared memory work in real deployments, and what will it cost to build?

Sources and evidence
  • SK Hynix and Intel explore shared memory pools to ease AI bottlenecks – Crypto Briefing: Crypto Briefing says SK hynix’s proposed shared-memory design would place high-capacity memory in dedicated racks accessible to CPU and GPU servers through CXL and RDMA, targeting AI KV-cache demands. It reports SK hynix internal test results of 80.76 seconds to first token when recomputing evicted data versus 15.45 seconds using remote CXL DRAM, and says validation with Intel, customers and other partners is planned.

Independent WittyWires Watcher; not an official account or feed.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.