Google adds secure inference and bigger context capacity to its TPU range

Google has added two systems to its eighth-generation TPU range: one designed for confidential inference, the other for workloads needing more compute and memory bandwidth. The practical shift is a more specialised AI infrastructure menu, with security and large context windows getting dedicated hardware treatment.

Google DeepMind Watch analysis

What happened

The TPU 8tx is aimed at organisations running proprietary models and autonomous agents on sensitive data. Google AI and Infrastructure chief technologist Amin Vahdat says it protects data and model weights in memory during processing, and offers up to twice the latency reduction of TPU 8t with a near-zero performance penalty. It uses fifth-generation AMD EPYC host processors.

The TPU 8i Max combines two TPU 8i chips on one package. Google says it doubles the compute, on-chip SRAM and HBM bandwidth of TPU 8i, while fitting within the existing rack footprint. Electronics Weekly reports that the systems join the TPU 8t and TPU 8i, introduced in April.

Why it matters

The announcements point to two distinct pressures in AI infrastructure: keeping sensitive information protected while models run, and feeding larger workloads without leaving accelerators waiting for data. These are concrete design aims, not proof that every workload will see the stated gains. But the new options give operators a clearer choice between training, real-time reasoning, confidential inference and memory-hungry tasks.

Our read

More specialised chips are useful only if they solve a problem customers actually have. The 8tx’s in-use protection is the more consequential promise for organisations handling sensitive data; the 8i Max’s bandwidth and rack-density claims will matter most to teams hitting memory limits. Google is making a credible case for a broader toolkit, and the next test is whether access, software support and workload results match the architecture.

What to watch

• When Google makes the new systems available to customers.
• Which workloads can use the 8tx’s confidential-computing features.
• Whether independent workload results support Google’s latency and capacity claims.

Discussion spark: For AI workloads handling sensitive data, would you prioritise hardware-level protection during processing or the largest possible compute and memory capacity?

not affiliated with, endorsed by, or operated by Google or Google DeepMind