HBM (high-bandwidth memory)

HBM (high-bandwidth memory) is DRAM stacked vertically into towers and wired directly beside a processor, delivering terabytes per second of memory bandwidth. It is the memory inside modern AI accelerators like Nvidia GPUs. The market is led by SK Hynix, with Micron and Samsung competing, as the industry moves to HBM4.

What HBM is, in plain terms

HBM (high-bandwidth memory) is a type of computer memory (DRAM) built by stacking several memory chips vertically into a tall "tower" and placing that tower right next to a processor such as a GPU. Ordinary PC memory sits in flat sticks centimeters away from the chip; HBM instead packs many layers into a tiny footprint sitting millimeters from the processor. The result is a very wide, very short data highway. A single HBM stack can move on the order of **1-2 terabytes per second**, far beyond conventional DDR or GDDR memory, which is why HBM has become the default memory for AI and high-performance computing.

How it works: stacking, TSVs and 2.5D packaging

An HBM stack is made of a **base logic die** at the bottom with 4 to 16 **DRAM dies** stacked on top. The layers are connected vertically by thousands of microscopic copper channels drilled through the silicon, called **through-silicon vias (TSVs)** — essentially elevators carrying data and power straight up the tower instead of routing it around the edges. The stacks don't sit on the processor itself. Instead, the GPU die and several HBM stacks are mounted side by side on a thin slab of silicon called an **interposer**, which provides ultra-dense wiring between them. This arrangement is called **2.5D packaging**, and TSMC's version of it is branded **CoWoS** (Chip-on-Wafer-on-Substrate). Co-locating the memory next to the logic is what makes the interface so wide (HBM4 uses a 2,048-bit interface) while keeping power per bit low.

Why it matters for AI and data centers

Training and running large AI models is bottlenecked less by raw compute than by how fast data can be fed to the processor — the so-called "memory wall." Large language models have billions of parameters that must be streamed through the chip constantly, so memory **bandwidth** and **capacity** directly limit performance. HBM solves both: it provides huge bandwidth and large per-package capacity in a power-efficient form. That is why every leading AI accelerator — Nvidia's data-center GPUs, AMD's Instinct line, and various custom AI chips — uses HBM rather than standard memory. HBM is now one of the most supply-constrained, highest-margin components in the entire AI build-out.

Where HBM sits in the supply chain

HBM ties together several specialized industries. **Memory makers** fabricate and stack the DRAM. A **logic/base die** at the bottom of each stack is increasingly made by a foundry — Micron and SK Hynix have moved toward having **TSMC** manufacture advanced HBM4/HBM4E base dies on leading-edge nodes, turning HBM into a semi-custom subsystem. The finished stacks then go to **advanced packaging** (CoWoS-style 2.5D assembly), where capacity at TSMC and others has been a recurring bottleneck. Finally the packaged module ships to chip designers like Nvidia and AMD. So a single AI GPU depends on memory vendors, a foundry, and an advanced-packaging line all at once.

Who the key players are

Three companies make essentially all the world's HBM. **SK Hynix** is the clear leader and Nvidia's primary supplier — it held roughly **60%+ of the HBM market in 2025** and reportedly supplies the large majority of Nvidia's HBM. **Micron (MU)** — the only US-based maker — has gained ground and overtook Samsung for the #2 spot in HBM shipments during 2025. **Samsung** is the third major supplier and is working to regain share with HBM4. Because South Korea hosts two of the three makers, the broader Korean tech complex (tracked by the iShares MSCI South Korea ETF, **EWY**, which includes SK Hynix and Samsung) is closely tied to the HBM cycle. Supporting players include foundry/packaging leader TSMC and equipment vendors.

What's changing now: HBM4 and custom stacks

The industry is transitioning from **HBM3E** to **HBM4**, whose JEDEC standard was finalized in 2025. HBM4 doubles the interface width to 2,048 bits and pushes per-stack bandwidth toward and beyond 2 TB/s, with capacities of up to 48 GB per stack. Nvidia's next-generation **Rubin** platform is designed around HBM4, using eight stacks per GPU for hundreds of gigabytes of memory and tens of terabytes per second of aggregate bandwidth. The bigger structural shift is **customization**: with HBM4/HBM4E the base logic die is increasingly tailored to each customer (extra SRAM, compression engines, tuned signaling) and built on advanced foundry nodes — moving HBM away from being a commodity and toward a co-designed part of the accelerator.

Frequently asked

What does HBM stand for?

HBM stands for high-bandwidth memory. It is a JEDEC-standardized type of DRAM that stacks multiple memory chips vertically and places them right beside a processor to deliver very high memory bandwidth.

How is HBM different from regular DDR or GDDR memory?

DDR and GDDR are flat memory chips connected over relatively narrow buses some distance from the processor. HBM stacks chips into 3D towers connected by through-silicon vias and mounts them millimeters from the GPU on a shared interposer, giving a much wider interface, far higher bandwidth, and better energy efficiency per bit — at higher cost and complexity.

Why is HBM so important for AI?

AI models are limited by how fast data can be moved to the processor (the memory wall), not just raw compute. HBM provides the enormous bandwidth and capacity needed to keep AI accelerators fed, so it is the standard memory in nearly all data-center AI chips.

Which companies make HBM?

Only three suppliers make HBM at scale: SK Hynix (the market leader and Nvidia's main supplier), Micron, and Samsung. TSMC plays a growing role manufacturing the logic base dies and doing the advanced 2.5D packaging.

What is HBM4 and how is it better?

HBM4 is the latest generation, standardized in 2025. It doubles the interface to 2,048 bits, pushes per-stack bandwidth toward and beyond 2 TB/s, raises capacity, and allows customizable base dies. Nvidia's upcoming Rubin GPUs are built around HBM4.

Is HBM a good way to invest in the AI memory trend?

HBM is a component, not a tradable asset. Investors typically get exposure through memory makers such as Micron (MU) or through broad Korea-tech funds like the EWY ETF that hold SK Hynix and Samsung. This is general information, not investment advice.

Related companies

$MU $EWY