NVIDIA data centre GPUs
NVIDIA Data Centre GPUs: H100, H200, Blackwell and Vera Rubin
How NVIDIA’s data-centre chips differ in architecture, memory, interconnect and performance, from Hopper to Blackwell to Vera Rubin, and what each means for the data centre you build around it.
Export controls: AI chips, servers, networking and memory can need an export licence depending on destination, end user and end use under US, UK and EU rules. We screen every project for this. See what applies.
The three levers that change every generation
- Compute Tensor cores do the matrix maths behind AI. Each generation adds lower-precision formats (FP8 in Hopper, FP4 in Blackwell, NVFP4 refined in Rubin) that perform more operations per second and per watt, when the model tolerates the precision.
- Memory Inference is often limited by how fast model weights and the context cache can be read, not by arithmetic. HBM capacity and bandwidth decide which models fit on a GPU and how quickly it produces tokens.
- Interconnect Large models are split across many GPUs. NVLink links GPUs far faster than ordinary networking, and rack-scale systems make dozens of GPUs behave as one large accelerator.
Generation by generation
- Hopper H100 (2022). A single large die on TSMC 4N with about 80 billion transistors, 80 GB of HBM3 at 3.35 TB/s, 132 streaming multiprocessors and 16,896 CUDA cores. Its Transformer Engine mixes FP8 and 16-bit maths to speed up transformer models. NVLink 4 gives 900 GB/s per GPU, at up to 700 W. It powered the first wave of large language model clusters.
- Hopper H200 (2024). The same Hopper compute with 141 GB of HBM3e at 4.8 TB/s: 76% more capacity and 43% more bandwidth than H100 at a similar 700 W class. It adds no new compute; it removes the memory bottleneck. NVIDIA reports up to 1.9x faster Llama 2 70B inference and 1.6x faster GPT-3 175B inference than H100, in preliminary tests.
- Blackwell B200 and GB200 (from late 2024). Two reticle-sized dies in one package, joined at 10 TB/s, on TSMC 4NP with about 208 billion transistors. Up to 192 GB of HBM3e at up to 8 TB/s, a second-generation Transformer Engine with FP4 and microscaling formats, and NVLink 5 at 1.8 TB/s per GPU. GB200 pairs two GPUs with a Grace CPU, and the NVL72 rack links 72 GPUs and 36 CPUs in one NVLink domain, at roughly 120 kW and liquid cooled.
- Blackwell Ultra B300 and GB300 (from 2025). The Blackwell design tuned for reasoning-model inference: 288 GB of HBM3e, about 15 PFLOPS of dense FP4 per GPU against roughly 10 on B200, and up to about 1.4 kW per GPU in GB300. Reports indicate FP64 throughput was cut sharply, so it targets AI rather than double-precision HPC.
- Vera Rubin (2026 onward). Two compute dies on a 3 nm class process (reported) with about 336 billion transistors, up to 288 GB of HBM4 at roughly 19 to 22 TB/s, NVLink 6 at 3.6 TB/s per GPU, and NVFP4 compute of up to 50 PFLOPS for inference. It pairs with the Vera CPU over NVLink-C2C at 1.8 TB/s. The rack is Vera Rubin NVL72: 72 GPUs, 36 Vera CPUs and about 20.7 TB of HBM4. NVIDIA claims up to 10x lower inference cost per token than Blackwell on a specific benchmark, which is a vendor claim to test against your own workload.
Side by side
| H100Hopper | H200Hopper | B200 / GB200Blackwell | B300 / GB300Blackwell Ultra | Rubin / Vera RubinVera Rubin | |
|---|---|---|---|---|---|
| Status | In volume since 2022 | Shipping since 2024 | Shipping since late 2024 | Shipping since 2025 | Ramping: first production shipments reported Aug 2026 |
| Process and size | TSMC 4N, about 80B transistors | Same Hopper die | TSMC 4NP, about 208B transistors, two dies | Same Blackwell design, two dies | 3 nm class (reported), about 336B transistors, two dies |
| Memory | 80 GB HBM3 | 141 GB HBM3e | Up to 192 GB HBM3e | 288 GB HBM3e | Up to 288 GB HBM4 |
| Memory bandwidth | 3.35 TB/s | 4.8 TB/s | Up to 8 TB/s | Up to 8 TB/s | About 19 to 22 TB/s (NVIDIA figures vary) |
| NVLink per GPU | 900 GB/s (NVLink 4) | 900 GB/s (NVLink 4) | 1.8 TB/s (NVLink 5) | 1.8 TB/s (NVLink 5) | 3.6 TB/s (NVLink 6) |
| Lowest precision | FP8 | FP8 | FP4 | FP4 (NVFP4) | NVFP4 |
| Peak low-precision compute per GPU | FP8: about 2 PFLOPS dense, 4 sparse | FP8: about 2 PFLOPS dense, 4 sparse | FP4: about 9 to 10 PFLOPS dense, 18 to 20 sparse | FP4: about 15 PFLOPS dense | NVFP4: 35 PFLOPS dense (training), up to 50 (inference) |
| Board power | Up to 700 W | Up to 700 W | About 1,000 to 1,200 W | Up to about 1,400 W | Analyst estimates of 1.8 to 2.3 kW, not confirmed by NVIDIA |
| CPU pairing | x86 hosts, or Grace in GH200 | x86 hosts, or Grace in GH200 | Grace (GB200) | Grace (GB300) | Vera, linked at 1.8 TB/s |
| Rack-scale system | HGX and DGX 8-GPU servers | HGX and DGX 8-GPU servers | GB200 NVL72: 72 GPUs, 36 CPUs, about 120 kW, liquid cooled | GB300 NVL72, same rack envelope | Vera Rubin NVL72: 72 GPUs, 36 CPUs, 20.7 TB HBM4 |
Peak figures are vendor numbers per GPU. Some are preliminary, some differ between NVIDIA documents, and sparse figures assume structured sparsity. Treat them as direction, not a guarantee.
How to read the performance numbers
- Sparse is not dense. Headline peaks often assume 2:4 structured sparsity, which doubles the figure. Compare dense with dense.
- Precision must match your model. FP8, FP4 and NVFP4 numbers apply only if your model runs at that precision. Many workloads still run in 16-bit, where the gains are smaller.
- Per GPU is not per rack. Rack-scale NVLink domains change what is possible for very large models and long contexts. Compare cost per token or time to train on your own workload, not peak FLOPS.
- Vendor claims are a starting point. The H200 and Rubin gains quoted above come from NVIDIA tests on chosen models. Our assessment turns them into numbers for your workload.
What it means for your data centre
- Power and cooling. Board power moves from about 700 W (Hopper) to 1,000 to 1,400 W (Blackwell) and, by analyst estimates, above 1.8 kW for Rubin. At roughly 120 kW per NVL72 rack, liquid cooling decides when you can deploy. See the rack power and cooling guide.
- Networking. Inside the rack NVLink carries the load. Between racks you still need InfiniBand or Ethernet fabrics with fast NICs. See AI networking.
- Memory supply. The move from HBM3e to HBM4 ties GPU availability to memory supply. See memory suppliers.
- Security. Rack-scale systems concentrate a lot of value and access. We design and review them under our agent security and defence practice.
- Export rules. Which parts you can buy, and where they can run, depends on the chip, the destination and the end user. We screen every project. See export controls.
Which generation fits which job
- H100 and H200. Proven, with the widest software support and plenty of hosted capacity. H200 suits memory-bound inference of large models. Power and cooling needs are the most modest of the three generations.
- Blackwell and Blackwell Ultra. Large-scale training and high-throughput reasoning inference, as 8-GPU servers or NVL72 racks. Plan the facility first: power, liquid cooling and rack weight.
- Vera Rubin. For deployments planned for 2027, with early access only now. Allocation will be tight, so design the facility and secure the supply chain early.
Not sure which fits? Try the sizing wizard, compare platforms, or rent versus own.
The NVIDIA product families
- HGX and DGX. 8-GPU server platforms: HGX is the board partners build servers around, and DGX is NVIDIA’s own system.
- NVL72 racks. GB200, GB300 and Vera Rubin NVL72: 72 GPUs and 36 CPUs as one NVLink domain, liquid cooled.
- MGX. A modular server reference design for mixing GPUs, CPUs and networking in different chassis.
- Networking. NVLink and NVSwitch inside the rack, InfiniBand and Spectrum-X Ethernet between racks, with ConnectX SuperNICs and BlueField DPUs.
- Software. CUDA, NVIDIA AI Enterprise and NIM inference microservices. Our NVIDIA page covers the stack.
CUDA: the platform layer that ties the hardware together
CUDA is NVIDIA’s software platform for GPUs: the compiler, runtime, drivers and libraries that applications and AI frameworks run on. Every chip above is reached through it, which makes CUDA a platform utility you plan and maintain like any other part of the stack.
- What it provides. A compiler and runtime, driver and memory management, GPU libraries such as cuBLAS for matrix maths, NCCL for multi-GPU communication and cuDNN for neural-network primitives, plus profiling tools such as Nsight.
- How it maps to the generations. Hopper is compute capability 9.0 and Blackwell data-centre parts are 10.x. Hopper brought FP8 through the Transformer Engine, Blackwell brought FP4, and NVIDIA announced CUDA 13.5 on 4 November 2026 with production support for Vera Rubin, including NVFP4 matrix-multiply tuning.
- What is new in CUDA 13.5. NVIDIA highlights tile-based multi-GPU communication over NVLink, MPS version 3 for sharing one GPU between workloads, and Windows on Arm support.
- Why it matters when you buy. PyTorch, JAX and most inference engines are built on CUDA, so it is the largest GPU software ecosystem. That is a reason to choose NVIDIA and also a switching cost to weigh against AMD ROCm and Intel oneAPI. See the platform comparison.
- Version and driver discipline. New GPUs need a recent toolkit and driver. Blackwell needs CUDA 12.8 or later, and Rubin needs 13.5. Plan toolkit, driver and framework upgrades together, and test before you roll them across a cluster.
- Keeping it secure. Drivers, runtimes and container images are part of your attack surface. Patching them is the maintain stage of our agent security and defence practice.
We deploy and integrate the CUDA stack as part of installation and deployment and keep it current under maintenance and support.
The roadmap beyond Rubin
- Rubin Ultra (planned for 2027). Four compute chiplets per package, HBM4E at about 1 TB per package, NVLink 7 and the Kyber rack.
- Feynman (planned for 2028). Die stacking and custom HBM, with the Rosa CPU, according to reporting from NVIDIA’s March 2026 GTC.
- Accelerators beyond GPUs. NVIDIA has also said it plans to pair LPU accelerators with Vera Rubin for inference decode, as reported from GTC 2026. Rubin CPX, announced in September 2025, was reportedly deprioritised.
Everything beyond Rubin is a plan, not a shipping product, and plans change.
Where this fits in the stack
Common questions
What is the difference between the H100 and the H200?
Both use the Hopper architecture and similar compute. The H200 adds 141 GB of faster HBM3e memory at 4.8 TB/s, against 80 GB of HBM3 at 3.35 TB/s on the H100, which mainly speeds up memory-bound inference of large models. NVIDIA reports up to 1.9x faster Llama 2 70B inference in preliminary tests.
What changed from Hopper to Blackwell?
Blackwell uses two dies per package, adds FP4 and a second-generation Transformer Engine, raises memory to up to 192 GB of HBM3e at up to 8 TB/s, and doubles NVLink bandwidth to 1.8 TB/s per GPU. It also introduced the liquid-cooled NVL72 rack that links 72 GPUs in one NVLink domain.
What is Vera Rubin and is it available?
Vera Rubin is NVIDIA’s platform after Blackwell: the Rubin GPU with HBM4 and NVLink 6, paired with the Vera CPU. NVIDIA reported production shipments began in August 2026 and cloud providers announced early access from the end of September 2026. Broad availability is still ramping, so confirm lead times with us before you plan around it.
Why was NVL144 renamed NVL72?
NVIDIA moved to counting GPU packages instead of the dies inside them. A Vera Rubin rack has 72 packages, which earlier naming counted as 144 dies.
Do Blackwell and Rubin need liquid cooling?
Rack-scale NVL72 systems are liquid cooled by design. Some 8-GPU server designs offer air-cooled options in earlier generations, but power per GPU keeps rising, so confirm cooling requirements per system before you commit to a facility.
What is CUDA and do I need it?
CUDA is NVIDIA’s GPU software platform: compiler, runtime, drivers and libraries. If you run NVIDIA GPUs you need a CUDA toolkit and driver that match your GPU generation. Blackwell needs CUDA 12.8 or later, and NVIDIA announced CUDA 13.5 on 4 November 2026 with production support for Vera Rubin.
Should I wait for Rubin?
It depends on your timeline, power and workload. Hopper and Blackwell are available now and well supported. Rubin offers more memory and bandwidth but with early supply and higher facility demands. An assessment compares the options on your numbers.
Can I buy these GPUs for any country?
No. Export rules depend on the chip, the destination, the end user and the end use under US, UK and EU regimes. We screen every project before we specify hardware.
Sources and dating
Figures are as of 17 November 2026, from NVIDIA’s H200 page and from secondary sources including Blackwell and Rubin on Wikipedia, Glenn K. Lockwood’s Blackwell Ultra notes, GPU Per Hour NVIDIA’s CUDA 13.5 post and Tom’s Hardware. Some figures are preliminary and some differ between sources, so we confirm against current NVIDIA datasheets when we specify a build. NVIDIA, H100, H200, Blackwell, Vera Rubin and related names are trademarks of NVIDIA Corporation. Embedica is independent and is not affiliated with or endorsed by NVIDIA.
Next steps
Related: NVIDIA AI Infrastructure · AI and GPU Computing Systems · Data Centre Infrastructure · Memory Suppliers · AMD · Intel
Planning a Hopper, Blackwell or Rubin build
Get the numbers for your workload
