Intel Gaudi

Intel Gaudi is a family of AI accelerators designed for training and serving large neural networks, most notably large language models. The line originates from Habana Labs, an Israeli chip designer Intel acquired in 2019, and has become Intel's principal answer to Nvidia's dominance of the data-centre AI market.^[1]

History

Intel had spent years building accelerator programmes — the Nervana neural network processor and the Movidius vision chips — before acquiring Habana Labs for roughly $2 billion in December 2019. Habana's Gaudi processor was already in commercial deployment at the time of the deal, and it quickly displaced Intel's other bets as the company's flagship AI silicon.^[1]

The first Gaudi chips were tuned for recommendation and training workloads. Gaudi 2, launched in 2022, moved to a 7 nm process and became the processor inside the Intel Gaudi 2 accelerator cards offered to cloud customers. Gaudi 3 was unveiled at Intel Vision in April 2024, built on a 5 nm process with roughly 1.8 petaflops of BF16 and FP8 matrix compute, 128 GB of HBM2e memory and 3.7 TB/s of memory bandwidth — about 4× the BF16 compute and 1.5× the memory bandwidth of Gaudi 2.^[2][3]

Architecture

Each Gaudi accelerator combines two types of compute block: matrix multiplication engines (MMEs) for the dense linear algebra of deep learning, and tensor processor cores (TPCs) — programmable vector units — for everything else. Gaudi 3 carries eight MMEs and 64 TPCs, a heterogeneous design that lets a single chip handle training, fine-tuning and inference without leaning on a separate co-processor.^[2][3]

The defining feature of the architecture, however, is networking. Every Gaudi chip integrates 24 high-speed RoCE Ethernet ports directly into the die (1200 GB/s of bidirectional bandwidth on Gaudi 3), so clusters scale out over standard Ethernet rather than a proprietary interconnect such as Nvidia's NVLink or InfiniBand.^[3] Intel's pitch is therefore not raw speed but openness and cost: buy standard networking gear, avoid a single-vendor fabric, and pay less per unit of compute.

Software is delivered through the SynapseAI stack, which plugs into PyTorch and offers containers and models through Hugging Face. Intel has also shipped reference systems — an eight-accelerator baseboard and a 32-node cluster design — so buyers can deploy Gaudi without designing the surrounding hardware themselves.^[4]

Market position

Gaudi's commercial record is mixed. Intel set an ambitious revenue target for the line in 2024 and later conceded it fell short, with hyperscale customers such as Microsoft and Amazon choosing their own chips or staying with Nvidia. Intel has responded by making Gaudi available through OEMs, channel partners and cloud providers rather than direct sales alone, and by positioning the cards — in both PCIe and OAM mezzanine form factors — as a pragmatic option for enterprise inference and fine-tuning where Nvidia availability or pricing is a constraint.^[1][4]

The strategic importance of the line is larger than its unit sales: it is the clearest evidence that a credible non-Nvidia training and inference accelerator can be built on commodity Ethernet, a theme that also underpins AMD's Instinct line and the cloud providers' internal silicon.

>See Also

• Intel
• Nvidia
• Cerebras Systems
• Google TPU
• AI Data Centres in Malaysia
• Malaysia AI Talent