HBM — High Bandwidth Memory Chips

High Bandwidth Memory is the memory architecture that makes large-scale AI training possible at current GPU compute densities. By stacking multiple DRAM dies vertically, connecting them with through-silicon vias (TSVs), and integrating the stack directly onto the same package substrate as the GPU via TSMC's CoWoS 2.5D packaging, HBM delivers bandwidth that conventional DRAM interfaces cannot approach at acceptable power. An NVIDIA H100 SXM5 paired with six HBM3 stacks achieves approximately 3.35 TB/s of memory bandwidth — roughly 65× the bandwidth of a dual-channel DDR5 desktop system. Without HBM, the matrix multiplications that define transformer-based LLM training would be memory-bandwidth-starved regardless of GPU arithmetic throughput.

HBM is also the single most acute supply chain chokepoint in AI infrastructure. SK Hynix is the dominant HBM3/HBM3E supplier. TSMC CoWoS packaging capacity is required to integrate HBM with GPU dies. Both are booked against AI GPU demand years in advance. The convergence of DRAM stacking yield, packaging capacity, and a single dominant customer relationship (NVIDIA-SK Hynix) at the same bottleneck makes HBM supply chain risk uniquely concentrated even within a semiconductor landscape defined by concentration.

HBM Chip Families — Generations, Specs & GPU Deployment

TSV Stack Architecture

A HBM stack begins with multiple DRAM dies fabricated on a standard DRAM process node (currently 1α/1β). Through-silicon vias — vertical electrical connections etched through the full thickness of each die — are formed and filled with copper. The dies are thinned to approximately 50 micrometers and stacked using thermal compression bonding with micro-bump interconnects at each layer interface. A logic base die at the bottom handles memory controller functions, PHY circuitry, power management, and ECC — and for HBM4, this base die may be fabricated on a leading-edge logic process at TSMC rather than on the same DRAM node as the memory dies, enabling higher controller performance and lower power.

Vendor Supply Position — The SK Hynix-NVIDIA Lock

SK Hynix established HBM3 production leadership ahead of Samsung and Micron, qualifying its product with NVIDIA for the H100 and securing supply allocations that gave it a near-exclusive position for the most strategically important AI GPU generation. This was not accidental: SK Hynix began HBM investment earlier and accepted near-term margin compression to build TSV and stacking process capability before demand materialized at scale. The NVIDIA supply relationship that resulted mirrors the structural lock-in seen in other high-specificity supply chains — NVIDIA's H100/H200/B200 GPU packages are physically designed around SK Hynix HBM geometry, and changing HBM suppliers requires full re-qualification of the packaged product.

Samsung's HBM3E qualification delays with NVIDIA — reportedly related to thermal performance under sustained AI workload conditions — extended SK Hynix's supply window through the H200 and into the Blackwell generation. Samsung has since qualified HBM3E and is competing for B200-generation allocation. Micron is positioning HBM3E as a supply diversification source for hyperscalers, particularly those seeking US domestic fab alternatives.

CoWoS — The External Packaging Bottleneck

HBM production at the memory IDM is only half of the supply chain. Before an HBM stack can reach an AI GPU, it must be integrated with the GPU die on a common silicon interposer using TSMC's CoWoS process. CoWoS is a TSMC-exclusive advanced packaging technology; its capacity is entirely separate from TSMC's wafer fab capacity. A CoWoS line requires dedicated equipment and facility space, and expanding it follows the same multi-year lead time as any semiconductor manufacturing capacity addition.

During the 2023–2024 AI GPU demand surge, CoWoS became the binding supply constraint on NVIDIA GPU shipments — not TSMC N4 wafer starts, not SK Hynix HBM production, but the packaging step that joins them into a shippable product. TSMC has been aggressively expanding CoWoS capacity across CoWoS-S (silicon interposer), CoWoS-L (silicon-less RDL interposer), and CoWoS-R (hybrid) variants, but the expansion timeline means this remains a structural bottleneck through Blackwell and early Rubin GPU generations. Full coverage of CoWoS architecture, capacity, and variants is on the Advanced Packaging page.

HBM4 — Logic Base Die Transition

HBM4 introduces the possibility of fabricating the base die on a leading-edge logic process (TSMC N3 or similar) rather than on the same DRAM node as the memory dies. Using a logic node for the base die enables significantly more sophisticated memory controller logic, higher PHY speeds, and power management capabilities not achievable on a DRAM process. This is structurally significant: it would introduce TSMC as a supplier of a HBM component that was previously entirely within SK Hynix or Samsung vertical integration, adding a new external dependency and a new coordination requirement — base die wafer starts at TSMC must align with DRAM die production at the IDM and CoWoS packaging capacity at TSMC simultaneously.

Supply Chain Bottleneck Summary

Related Coverage

Memory & Storage Overview | DRAM Supply Chain | NAND Flash Supply Chain | AI Inference & Edge Compute SoCs | AI Accelerators | CoWoS Advanced Packaging | Semiconductor Bottleneck Atlas | NVIDIA Spotlight

Cross-Network — ElectronsX Demand Side

Every HBM-equipped GPU shipped into a hyperscaler data center adds to the electrical load reshaping grid infrastructure planning, transformer procurement, and utility interconnection queues. The AI training cluster buildout driving HBM demand is the same infrastructure expansion driving data center power density and cooling demand covered across ElectronsX.

EX: ADAS/AV Compute Architecture | EX: Humanoid Robots | EX: Supply Chain Convergence Map