JEDEC: here's the standard that could reduce memory costs for artificial intelligence

JEDEC has published the specification JESD330-4, which introduces the SPHBM4 (Standard Package High Bandwidth Memory) standard, which we discussed last year. The goal is to reduce the production cost of HBM4 memory that powers the fastest AI accelerators on the market. The standard does not address the DRAM shortage, as it uses the same large HBM4 stacks, but it cuts integration costs by eliminating interposers and advanced packaging like TSMC's CoWoS.

The difference compared to traditional HBM4 lies entirely in the base die. Instead of the 2048-bit interface, SPHBM4 adopts a new PHY/buffer die with a 512-bit interface, organized into 32 independent 16-bit DDR channels, grouped into eight "Quad Channels." Each group of four 64-bit HBM4 channels is compressed into 64 data pins, operating at quadruple speed compared to the original HBM4 interface to compensate for the reduced width.

To maintain the bandwidth, SPHBM4 pushes the transfer rate to a range between 22.4 and 46.0 GT/s per pin. The DRAM core, however, remains identical to that of the starting HBM4: same architecture, same row activation times, same refresh. The core operates at a quarter of the frequency of the external interface, thus 2 GHz in the 32 GT/s bin. However, the new base die adds equalization, lane training, and FEC error correction, elements that introduce some additional nanoseconds of latency.

Cheaper but not free: the SPHBM4 standard comes to HBM4 RAM. The standard supports a bump pitch greater than 90 µm and a channel length of up to 20 mm, features that allow for the abandonment of the interposer and rely on routing on inexpensive organic substrates. However, this does not make SPHBM4 automatically affordable: HBM4 stacks, 2.5D packaging, through-silicon vias, and a more complex base die than the standard HBM4 are still required. The JESD330-4 specification provides stacks of 4, 8, 12, or 16 dies with densities of 24 or 32 Gb, up to a maximum of 64 GB per stack, the same limit as HBM4E.

On the performance front, an HBM4 at 8 GT/s delivers about 2 TB/s per stack, while HBM4E will reach 3-3.3 TB/s with transfer rates of 12-12.8 GT/s. SPHBM4, at the maximum speed of 46 GT/s, can reach 2.944 TB/s, but early implementations are unlikely to hit that limit. HBM4 and HBM4E will therefore maintain an advantage in both bandwidth and latency since the SPHBM4 interface still introduces a serialization phase absent in the parallel connections of classic HBM4.

On power consumption, the DRAM core shares the same voltage as HBM4 but changes the I/O management: while HBM4 leaves manufacturers the choice between 0.7V, 0.75V, 0.8V, or 0.9V, SPHBM4 sets the standard at 0.75V. The potential impact on Chinese AI accelerator producers like Biren, Huawei, and Moore Threads, who are excluded from TSMC's advanced packaging services, is interesting. A tighter interface and organic substrates fit better with their production base. The issue remains upstream: only Samsung, SK Hynix, and Micron can produce HBM4 stacks, while the Chinese CXMT is stuck at HBM2E, and developing a PHY capable of supporting 46 GT/s is not a trivial task.