Nvidia Presented an LPU It Says It Doesn't Make

Nvidia Presented a Groq LPU It Says It Does Not Make

315 PFLOPS, 128 GB of SRAM, 40 PB/s. Every one of those numbers is a rack of 256 chips, and the chip is somebody else's.

Aug 26, 2026Source: ServeTheHome

In session AI 1 at Hot Chips on Tuesday, two Nvidia engineers presented the Groq 3 LPU — and then explained that Nvidia does not build it. The parts going into Vera Rubin-generation racks are bought from Groq, whom Nvidia is absorbing in what the talk described as a broader acquihire.

The numbers, and their true denominator

A rack of 256 LPUs delivers 315 PFLOPS of FP8, carries 128 GB of SRAM and sustains 40 PB/s of aggregate SRAM bandwidth, running 11,000 decode tokens per second on a 31-billion-parameter model. Per-LPU TDP and process node were not disclosed.

What the common framing gets wrong

Every headline figure here is rack-scale and will be read as chip-scale. "128 GB of SRAM" sounds like an enormous on-die cache; divided by 256 it is roughly half a gigabyte per LPU, with no HBM at all — which is the known Groq trade-off, not a surprise, but it means a model must be sharded across hundreds of chips to fit. A 128 GB rack cannot hold a large model plus KV cache the way a single 216 GiB HBM4 part can. "40 PB/s" is the same arithmetic applied to bandwidth: the sum of 256 separate SRAM arrays, saying nothing about how fast data crosses between chips, which is the binding constraint in an SRAM-only design. And 315 PFLOPS is peak FP8, with the token-rate demo run on a 31B model — small by 2026 standards, and sized to fit the SRAM budget.

The second distortion is corporate

This is not "Nvidia's LPU." The talk says on the record that Nvidia does not currently produce them, that it has its own under development, and that it is purchasing these from Groq. Writing this up as an Nvidia product launch inverts what was actually said from the stage.

Why Nvidia would do this at all

Dense GPUs are the wrong shape for low-latency decode: the work is memory-bound, batch sizes are small, and HBM latency dominates. An SRAM-only architecture attacks exactly that. Nvidia standing on a conference stage presenting a competitor's answer — while buying the parts and hiring the team — is a clearer admission of the gap than any roadmap slide.

Why It Matters

Nvidia presenting somebody else's accelerator, and confirming it is absorbing that company while purchasing its chips, is the strongest signal yet that the market is splitting: dense GPUs for training, something else entirely for inference decode. Nvidia would rather own the alternative than compete with it.