SK hynix is proposing a shared memory pool for AI servers, putting memory in a separate rack that multiple processors can use. In an evaluation described by the company, remote CXL memory cut the time to generate the first token from 80.76 seconds to 15.45 seconds, a roughly 5.2-fold improvement over recomputing discarded data.
SK Hynix Watch analysis
What happened
At the Intel AI Summit Seoul on 12 November, SK hynix vice-president Joo Young-pyo outlined a design that separates memory from individual GPU servers. A dedicated rack would hold tens to hundreds of terabytes, connected to servers using RDMA between racks and CXL within a rack. Existing local memory would remain in place; the shared pool would add capacity rather than replace it.
The aim is to reduce delays and repeated calculations when AI systems run short of memory. In multi-turn conversations, the key-value cache that holds context from earlier turns keeps growing. If it has to be discarded, the system may need to calculate that information again. SK hynix is developing the concept with customers and partners, including Intel, according to BigGo Finance’s account of the presentation.
Key findings
- A shared memory tier
The proposed system adds a central pool alongside GPU and CPU server racks, rather than tying all memory to individual processors. - Faster than recomputation in one evaluation
SK hynix and its partners reported first-token generation time falling from 80.76 to 15.45 seconds when remote CXL DRAM was used. - CXL memory beat remote SSD cache loading
The presentation gave a 33.56-second loading time for cache stored on a remote SSD.
Why it matters
AI infrastructure is often discussed as a contest for more powerful chips. This proposal targets a less glamorous but consequential constraint: getting enough useful data to those chips quickly, without throwing away work when memory fills up. If it works at data-centre scale, shared memory could change how operators balance compute, storage and the movement of data.
The reported speed-up is striking, but it comes from an evaluation described in a company presentation, not evidence here of a deployed system or a broad independent comparison. The proposal still needs compatible specifications, software support and validation of latency and reliability at data-centre scale.
Our read
This is a meaningful architectural idea, not yet a ready-made fix for the GPU wall. The practical takeaway is that memory capacity and data movement may offer another route to more efficient AI inference, alongside buying more accelerators. The evaluation gives the idea a useful first number; real deployments will have to do the harder work of making that number repeatable.
What to watch
- Whether SK hynix and Intel publish the evaluation setup and further results.
- Whether CXL and RDMA specifications and supporting software mature for shared-memory deployments.
- Whether customer trials demonstrate reliable performance at data-centre scale. SK hynix’s shared-memory proposal aims to ease a growing AI bottleneck without simply adding more GPUs. In a company-described evaluation, remote CXL memory cut first-token generation time by about 5.2 times compared with recomputing discarded data. The design adds a separate pool of memory alongside GPU and CPU server racks. It is a promising architectural direction, though it still needs software support and large-scale testing. Our Forum story explains what the proposal changes and what remains unproven.
Discussion spark: If shared memory can reduce repeated AI computation, should data-centre operators prioritise this kind of infrastructure before buying more GPUs?
Sources and evidence
- SK Hynix Unveils 'Memory Pool' Concept to Break Through GPU Wall, Co-Designed With Intel – BigGo Finance (12 November 2026, 06:05 UTC)
Independent WittyWires tracker for public updates about SK Hynix. Not affiliated with or endorsed by SK Hynix; this is not an official account.