High Bandwidth Memory

High Bandwidth Memory (HBM) is a family of high-performance dynamic random-access memory in which individual memory dies are stacked vertically and connected through thousands of microscopic channels, delivering far more bandwidth than conventional memory modules while using less energy per bit of data transferred. HBM is the memory technology used by the graphics processing units and other accelerators that train and serve large artificial intelligence models, and demand for it has made its three manufacturers — SK hynix, Samsung and Micron — strategic suppliers to the global AI industry.^[1][2]

Background

HBM was developed to address the "memory wall": the growing gap between the speed at which processors compute and the speed at which conventional DRAM can feed them data. The technology was standardised by JEDEC, the semiconductor industry body, and shipped commercially from 2015, initially in high-end graphics cards before finding its defining application in AI accelerators. Each generation roughly doubled capacity and bandwidth. HBM2 (2016) and HBM2E (2019) supported successive generations of data-centre GPUs; HBM3 arrived with the accelerators that powered the generative AI boom of the early 2020s; and HBM3E became the memory of choice for flagship AI chips in 2024 and 2025.^[1][2]

The cadence accelerated sharply with AI demand. HBM4, the current generation, doubles the interface width to 2,048 bits and raises per-stack bandwidth into the range of two terabytes per second and beyond, with capacities of up to 64 gigabytes per stack. It also introduces customisable logic base dies, allowing memory to be tailored to specific accelerator platforms — an architectural shift that has deepened co-engineering between memory makers and their largest customers.^[2]

Market and Suppliers

HBM is effectively a three-supplier market. SK hynix, the long-standing leader, entered HBM4 production first, and analysts estimated it would supply roughly 60 to 70 per cent of the HBM4 volume for Nvidia's Vera Rubin platform. Samsung, recovering from earlier yield problems, began mass production of HBM4 in February 2026 and holds an estimated quarter of the market. Micron began high-volume shipments of its HBM4 36GB 12-high product in the first quarter of 2026 and is expanding capacity in response to demand.^[3][5]

All three suppliers passed Nvidia's certification for Vera Rubin, which entered full production in mid-2026, and Nvidia's leadership publicly urged memory makers to increase output amid tight global supply. The imbalance between AI demand and memory capacity pushed HBM and conventional DRAM prices sharply higher through 2025 and 2026, making memory one of the largest cost components of AI servers and prompting new fabrication and packaging investments in South Korea, the United States and Singapore, where Micron is building an HBM advanced packaging facility due to contribute to supply from 2027.^[3][4][9]

Applications and Impact

HBM appears in every leading AI system, from Nvidia's Blackwell and Vera Rubin platforms to AMD accelerators and custom silicon such as Google's tensor processing units. Each accelerator carries several HBM stacks, and system memory capacity often limits how large a model can be served and how many users a data centre can support, which has made HBM supply a determinant of how quickly AI infrastructure — data centres, cloud regions and sovereign AI programmes — can be built out. The technology is also a substantial part of AI hardware cost: a single high-end AI accelerator can include several thousand dollars' worth of HBM.^[2][3]

The technology has reshaped semiconductor economics more broadly. Memory makers have diverted wafer capacity toward HBM, contributing to shortages of conventional memory, while equipment and materials suppliers have raced to support stacking heights of 12 and 16 dies and the complex packaging steps these require. Governments now treat advanced memory and packaging as strategic industries, subsidising new capacity under semiconductor resilience programmes in the United States, South Korea, Japan and the European Union.^[2][3][4]

>See Also

• NVIDIA
• NVIDIA Blackwell
• GPU Cluster
• AI Data Centres Malaysia
• Inference
• CUDA