The most striking claim of the Hot Chips 38 tutorial day on 23 August came from the smallest company presenting. d-Matrix described Raptor, a 3D-DRAM inference accelerator, and put its bandwidth density at about 32.6 GB/s per mm2 against roughly 1.5 GB/s per mm2 for HBM4 parts — a factor of about twenty.

The supporting numbers

Energy per bit on the 3D-DRAM I/O path is given as 0.37 pJ/bit, against 2.5-5 pJ for HBM4 systems, and power at 2.96 mW per GB/s against 40 mW for HBM alternatives. Capacity is 32GB per card, with a 72-card configuration described as hosting frontier models at 1M context and sustaining around 1000 tokens per second per user on a 3T-class model.

The wall it is aimed at

Micron had used an adjacent session to quantify the problem. Accelerator throughput grows roughly 3x every two years, while the bandwidth of 2.5D-attached memory such as HBM climbs at under 2x over the same period. Micron also put a typical package — GPU plus base die plus eight HBM4 instances — at over 12,000 mm2, with memory silicon able to exceed eight times the area of the GPU die itself at 12-high HBM4. That is the gap Raptor is pitched into.

What the common framing gets wrong

Every figure above is a vendor comparison presented by the vendor, in a tutorial session, with no shipping product attached. The disclosure contains no launch date, no availability timeline and no named customer. A 20x ratio on a single metric is not a 20x system: bandwidth density says nothing about yield, cost per bit, thermal behaviour in a real chassis, or whether the software stack exists. Comparisons of this shape are routinely repeated as though they were benchmarked results against shipping hardware. They are not — they are architectural claims, and the correct posture is to note them and wait for silicon.

Why it still matters

Because the incumbents agreed with the premise. Samsung, SK hynix and Micron each spent their slot on some version of the same constraint, and the fact that a challenger can propose a fundamentally different memory topology and get a hearing is itself the signal.