Memory Wall Deepens as Compute Outpaces HBM Bandwidth
At Hot Chips 2026, Micron's HBM Design Architecture Fellow, Raghu Sreeramaneni, presented a stark assessment of the challenges facing high-bandwidth memory (HBM). According to Micron, compute capabilities are scaling roughly threefold every two years, while HBM bandwidth advances less than twofold over the same period. This widening gap, which Micron describes as the 'memory wall,' threatens to become a major bottleneck for AI systems.
Micron highlighted that in a typical GPU system-in-package (SIP) with four 12-high HBM stacks, memory silicon accounts for approximately 90% of the total silicon—roughly eight times the area of the GPU itself. The company also pointed to Meta's Llama 3 training paper, which attributed 17% of unintended interruptions to HBM-related issues, underscoring the reliability stakes.
HBM vs. DDR: Trade-offs in Capacity and Bandwidth
Micron compared HBM with traditional DDR memory, noting that HBM delivers higher system bandwidth, making it the preferred choice for AI accelerators, while DDR offers higher capacities and takes up more space. The company argues that HBM's space and power efficiency justify its higher per-GB cost, a view echoed by SK hynix in a separate presentation.
SK hynix compared a typical GDDR6 solution—24 GB and 768 GB/s—with an HBM3E configuration using four stacks, which offers 144 GB capacity and 4 TB/s bandwidth while saving up to half the space. Both companies are positioning HBM as essential for scaling large language models and other memory-intensive workloads.
HBM4 Roadmap: Higher Bandwidth, More Complexity
Micron's HBM4 roadmap promises up to 2800 GB/s bandwidth, twice the I/O count, and double the channels of HBM3E. SK hynix presented similar specifications, including over 2 TB/s bandwidth, up to 48 GB capacity, 2048 I/O bits, and 8 Gbps I/O speeds. SK hynix detailed that HBM4 will feature more TSVs (through-silicon vias) and micro-bumps than HBM3E, with over 20,000 TSVs and 16,148 base micro-bumps on a 12.8x11 mm package with a 775-micrometer Z-height.
Stack heights have grown from 4-high to 16-high, with a path to 20-high, but SK hynix acknowledged that thermal and mechanical hurdles remain. The company noted that HBM4 capacity reaches 48 GB, with 12-high stacks in production and 16-high under qualification.
Packaging Trade-offs: TC+NCF vs. MR+MUF
A central theme of SK hynix's presentation, delivered by Jaesik Lee, was advanced packaging. The company outlined two main bonding approaches for stacking die-to-die: TC+NCF (thermal compression with non-conductive film) offers high productivity but is sensitive to warpage, while MR+MUF (mass reflow with molded underfill) handles thin-die warpage better but comes with higher thermal resistivity.
SK hynix described its process flow for manufacturing HBM, including full wafer-level processing and a known-good stacked die (KGSD) testing step. The company highlighted that achieving 16-high HBM3E at 48 GB per cube required halving chip thickness, gap height, and bump pitch—a packaging milestone.
Combatting Thermal Constraints
Both Micron and SK hynix identified thermal issues as a critical challenge, with Micron describing the problem as a power density issue, particularly affecting the base die. SK hynix introduced its i-HBM technology, which embeds a cooling component near the die-to-die PHY area to achieve more than a 30% reduction in thermal resistance.
Hybrid bonding is emerging as a key solution. SK hynix detailed that hybrid bonding uses pick-and-place at room temperature followed by an anneal above 200°C to form SiO2-to-SiO2 and Cu-to-Cu bonds. This technique allows for a 24% thicker core die and TSV pitch below 18 micrometers, with 35% lower thermal resistance compared to conventional methods.
Micron is also exploring fusion bonding and liquid cooling as part of its advanced packaging research, alongside high-speed I/O designs using a memory-optimized SerDes solution.
Competitive Approaches and Future Directions
SK hynix's presentation compared various packaging approaches, including TSMC's CoWoS-S and CoWoS-L, as well as Intel's EMIB technology. The company outlined three packaging direction tiers: die-on-die, die-to-die on interposer, and package-to-package on board, reflecting the industry's evolution toward more integrated designs.
Looking ahead, SK hynix is exploring 3D integration, potentially stacking HBM on top of accelerators. The company is also investigating ways to double TSVs and increase I/O speed through logic process integration, with optimized logic foundry processes and power TSVs to address power delivery network (PDN) challenges.
Micron, for its part, is refining base-die circuit design, while Samsung's HPB (High Bandwidth Processor) approach moves DRAM next to the processor with a copper heat spreader—illustrating different strategies across the industry.