Google Launches TPU 8t/8i AI Chips: Training Power Doubles, Inference Latency Reduced

Google unveiled two custom AI ASIC chips, TPU 8t and TPU 8i, at the Google Cloud Next conference. The TPU 8t is optimized for large-scale training, while the TPU 8i focuses on agent inference workloads—both can handle each other's key tasks.

The TPU 8t's Pod size increased from 9216 to 9600 chips, with FP4 computing power nearly doubling. Its single-chip scale-out bandwidth jumped from 100Gbps to 400Gbps, resolving bottlenecks in large-scale connectivity. The TPU 8i features a CAE on-chip acceleration engine, 384MB on-chip SRAM, and 288GB off-chip HBM. Its Pod size rose from 256 to 1152 chips, with per-Pod FP8 computing power and HBM memory increasing by 8.67x and 5.74x respectively.

Both chips are twice as energy-efficient as the previous-generation TPU Ironwood, enabling more efficient AI workload operations.