NVIDIA Rubin: 336 billion transistors, 288 GB of HBM4, and NVLink 6 for large-scale inference
With the growing prevalence of AI models capable of reasoning, planning, and executing complex sequences of operations, NVIDIA believes that data center infrastructure must evolve from a simple training platform to a true "intelligence factory." It is in this context that Rubin, the new GPU architecture intended for the Vera Rubin platform, designed to support the workloads of so-called agentic AI, fits in.
According to the company, modern AI agents no longer just generate a response to a prompt but must process long chains of reasoning, consult external tools, verify intermediate results, and manage large contexts. All of this increases the importance of latency in inference, the ability to quickly process generated tokens, and efficiency in memory management and communication between GPUs.
From a hardware perspective, Rubin represents a significant step forward compared to Blackwell. The GPU integrates 336 billion transistors, 224 Streaming Multiprocessors, 896 Tensor Cores, and the third-generation Transformer Engine, capable of achieving up to 50 PFLOPS in sparse operations in NVFP4 format. NVIDIA claims that, at equal power consumption, the Vera Rubin platform can offer up to 10 times the throughput in agentic inference workloads compared to the Blackwell generation.
The architecture is built around two unified computing chips interconnected through the proprietary NV-HBI interconnect. Resources are organized into Graphics Processor Clusters (GPC) with a centralized L2 cache, while the GigaThread Engine coordinates the execution of loads, and MIG partitions allow the GPU to be divided into multiple virtual instances, enabling the simultaneous execution of different workloads.
One of the most relevant aspects is the memory subsystem. Rubin uses up to 288 GB of HBM4 memory via 12-layer stacks, with a maximum bandwidth of 22 TB/s, which is approximately 2.8 times higher than the Blackwell platform. The increased capacity allows for larger models, broader contexts, and larger KV caches to remain resident while the high bandwidth aims to support the token generation phase, where memory often represents the primary bottleneck.
The Tensor Memory Accelerator has been updated to reduce the cost of data movement, particularly in Mixture of Experts (MoE) models, where the high number of experts complicates memory descriptor management. Rubin introduces a system that allows a single descriptor to be shared among multiple tensors, dynamically modifying only the necessary information during execution, reducing overhead and increasing the time dedicated to actual processing.
The computing subsystem has also been optimized. The Tensor Cores double the amount of data processed along the K dimension of matrix operations, decreasing the number of iterations needed during matrix multiplications distributed across many GPUs. For workloads with very large contexts, Rubin also introduces new sparse 2:4 compression techniques in attention operations, combined with an increase in the speed of the exponential operations used by softmax, up to four times faster than Blackwell in BF16 and FP16 formats.
NVIDIA has also worked on reducing latency between consecutive kernels, allowing consumer kernels to start processing as soon as the necessary data is available, without waiting for the completion of the entire producer kernel. The goal is to increase the number of tokens processed per second while maintaining high occupancy of the computing units.
Communication between GPUs is another central element of the architecture. NVLink 6 offers up to 3,600 GB/s of bandwidth for GPU-GPU communications, while NVLink-C2C reaches 1,800 GB/s between CPU and GPU. There is also a PCI Express 6.0 x16 interface with 256 GB/s towards the host system. Rubin also introduces a new mechanism for device-initiated communication, reducing the need for external synchronizations and lowering latency in data exchanges between accelerators.
The architecture has been designed with particular attention to energy efficiency. In Vera Rubin NVL72 systems, which integrate up to 72 GPUs within the same rack, NVIDIA adopts Intelligent Power Smoothing technology, capable of compensating for the rapid fluctuations in consumption typical of AI loads. According to the company, the system reduces average consumption by about 10% compared to the Grace Blackwell NVL72 racks and lowers power peaks over intervals of 50 milliseconds by 20%.
At the infrastructure level, DSX MaxLPS comes into play, a solution that coordinates power, liquid cooling, networking, and load distribution. NVIDIA claims that this technology allows for the installation of up to 40% more GPUs while keeping the available energy budget unchanged, thereby increasing computational density without requiring an increase in installed electrical power. The entire Vera Rubin NVL72 platform is therefore configured as an infrastructure designed to deploy ever-larger AI models across domains composed of multiple racks, with a particular focus on performance, reliability, and operational efficiency.