NVIDIA has simply introduced its next-gen Hopper GPU structure, with the brand new flagship NVIDIA H100 GPU rocking 80 billion transistors, next-gen HBM3 reminiscence know-how, PCIe 5.0 connectivity, and sufficient energy that should you use 20 of NVIDIA H100 GPUs you’d have the ability to maintain the whole world’s web visitors.
VIEW GALLERY – 3 IMAGES
Yeah, you learn that proper: 20 x NVIDIA H100 GPUs have virtually 5TB/sec of exterior connectivity, in addition to 3TB/sec of reminiscence bandwidth because of its HBM3 reminiscence, which is sufficient that within the phrases of NVIDIA that 20 x NVIDIA H100 GPUs “can maintain the equal of the whole world’s web visitors“.
In the midst of that is the brand new Transformer Engine — which, when in comparison with the previous-gen A100 — the brand new NVIDIA H100 hurries up pure language processing (Transformer being one of the vital necessary deep studying fashions ever invented) by as much as a gigantic 6x efficiency with out dropping accuracy. Lovely work, NVIDIA… as at all times, hopping over the competitors (which is your self at this degree).
The brand new Hopper GPU structure from NVIDIA was named after pioneering US laptop scientist, Grace Hopper, succeeding the Ampere GPU structure. We’ve got 80 billion transistors on the NVIDIA H100, which now turns into the world’s largest and strongest accelerator.
NVIDIA and TSMC (Taiwan Semiconductor Manufacturing Firm) labored on the TSMC 4N course of node, tweaking it for the Hopper-based H100 GPU. There’s PCIe 5.0 help, next-gen ultra-fast HBM3 reminiscence know-how with an absolute blistering 3TB/sec (3000MB/sec). Madness.
However there’s additionally 4th-Technology NVIDIA NVLink know-how right here too, the place you need to use a brand new exterior NVLink Swap to increase NVLink as a scale-up networking past the server. You may join a bonkers 256 x NVIDIA H100 GPUs at 9x larger bandwidth versus previous-gen GPUs utilizing NVIDIA HDR Quantum InfiniBand.