Google Tensor Processing Unit

The Tensor Processing Unit (TPU) is an application-specific integrated circuit (ASIC) designed by Google specifically for machine learning workloads, replacing general-purpose GPUs with a systolic array architecture optimised for matrix multiplication — the arithmetic that dominates neural network computation. First used inside Google's own data centres in 2015 and offered to cloud customers since 2018, the TPU is the foundational hardware behind Google's search ranking, translation, and the training and serving of its Gemini model family.^[1]

History

Google publicly described the first TPU at the 2016 ISCA architecture conference: a 28 nm chip with 28 MiB of on-chip memory and 23 tera-ops of throughput, deployed since 2015 to accelerate the inference side of workloads such as DeepMind's AlphaGo and Google Translate.^[1] Subsequent generations followed roughly annually — v2 (2017) added GPU-like programmability and HBM memory, v3 (2018) and v4 (2021) scaled interconnect bandwidth with optical circuit switching, and v5e/v5p (2023) split the line into efficiency and performance tiers.^[1]

The sixth generation, Trillium (v6e), announced at Google I/O in May 2024, delivered a claimed 4.7× performance-per-chip improvement over v5e with doubled HBM capacity and bandwidth, and pods of up to 256 chips.^[2] The seventh generation, Ironwood (v7), unveiled at Google Cloud Next in April 2025, was Google's first TPU designed specifically for inference: 192 GB of HBM per chip (6× Trillium), native FP8 compute at 4,614 TFLOP/s per chip, and clusters of 9,216 liquid-cooled chips reaching 42.5 exaFLOPS per pod.^[3][4] An eighth generation (v8t for training, v8i for inference) followed in 2026.^[1]

Key Concepts

• Systolic arrays. Data flows through a grid of multiply-accumulate units in a rhythmic, wave-like pattern, keeping arithmetic units busy without repeatedly reading from memory — far more energy-efficient than a general GPU's fetch-execute cycle for matrix work.^[1]
• Inter-chip interconnect (ICI). TPUs are wired into multi-dimensional torus topologies inside a pod, so thousands of chips act as one machine; Ironwood pods reach 9,216 chips with optical circuit switching.^[3]
• Software stack. TPUs run the XLA compiler with JAX, TensorFlow and PyTorch (via TorchXLA), and cloud offerings now map onto serving frameworks such as vLLM.^[2][4]
• SparseCore. Dedicated units for embedding-heavy workloads such as ranking and recommendation, present since v4.^[4]

Applications and Impact

TPUs train and serve Google's own products — Search, Ads, Translate and Gemini — and are rentable by external customers as TPU VMs and multi-chip pods on Google Cloud. Ironwood's inference-first design reflects the industry shift: as generative AI moved from training races to serving billions of daily requests, accelerator design optimised for throughput-per-watt at inference became commercially decisive.^[3] The TPU line also gave the industry a credible alternative to NVIDIA's dominance, alongside AMD Instinct, AWS Trainium, Microsoft Maia, Groq and Cerebras.^[1]

>See Also

• NVIDIA
• Cerebras Systems
• Groq
• AI Data Centres in Malaysia