Aegis-GPU2.9 High-speed network options
NVLink and NVLink Switch inside a system; InfiniBand and Spectrum-X Ethernet between systems.
Key points
NVLink is NVIDIA's direct GPU-to-GPU interconnect. It scales multi-GPU input and output within a server. NVIDIA lists 1.5x higher bandwidth per GPU at 900 GBps with fourth-generation NVLink on DGX H100. Newer generations are faster, so always check which generation a question means.
What NVIDIA says (2)
“is a direct GPU-to-GPU interconnect that scales multi-GPU input/output (IO) in the server.”
“1.5X higher bandwidth per GPU @ 900 GBps with fourth generation of NVIDIA NVLink.”
Point-to-point links only connect pairs. A switch lets every GPU reach every other GPU. NVIDIA says NVLink Switch chips connect multiple NVLinks to provide all-to-all GPU communication at full NVLink speed. NVLink Switches also include SHARP engines for in-network reductions.
What NVIDIA says (3)
“which connects multiple NVLinks to provide all-to-all GPU communication at the total NVLink speed.”
“The NVIDIA NVLink Switch chips connect multiple NVLinks to provide all-to-all GPU communication at full NVLink speed across the entire rack.”
“To enable high-speed, collective operations, each NVLink Switch has engines for NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)™ for in-network reductions and multicast acceleration.”
Multi-tenant means many customers share the same infrastructure. 'Noise' is one tenant's traffic slowing another. NVIDIA says Spectrum-X Ethernet is designed for cloud providers and large enterprises running multi-tenant AI at hyperscale. It uses telemetry-based congestion control for noise isolation, and SuperNICs provide RoCE (RDMA over Converged Ethernet) between GPU servers.
What NVIDIA says (4)
“NVIDIA Spectrum-X is an AI-optimized Ethernet networking platform that combines NVIDIA Spectrum switches with the BlueField-3 SuperNIC”
“NVIDIA Spectrum-X Ethernet is designed for cloud service providers and large enterprises running multi-tenant AI workloads at hyperscale.”
“Spectrum-X Ethernet uses advanced telemetry-based congestion control to provide noise isolation between tenants.”
“the BlueField-3 SuperNIC provides best-in-class remote direct-memory access over converged Ethernet (RoCE) network connectivity between GPU servers”
SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) lets switches do part of a collective operation, so less data crosses the network. Adaptive routing sends traffic around busy links. NVIDIA highlights SHARP, quality-of-service features including congestion control and adaptive routing, and self-healing networking for Quantum InfiniBand.
What NVIDIA says (4)
“technology improves the performance of MPI and Machine Learning collective operation, by offloading collective operations from CPUs and GPUs to the network”
“This innovative approach decreases the amount of data traversing the network as aggregation nodes are reached”
“NVIDIA Quantum InfiniBand is the only high-performance interconnect solution with proven quality-of-service capabilities, including advanced congestion control and adaptive routing, resulting in unmatched network efficiency.”
“NVIDIA Quantum InfiniBand with self-healing network capabilities overcomes link failures”
Key terms
- RDMA over Converged Ethernet: RDMA carried over Ethernet networks instead of InfiniBand.
- InfiniBand: A high-performance, low-latency, RDMA-capable network used to connect GPU servers and storage.
- SHARP: Switch-based in-network computing that performs collective operations so less data crosses the network.
- NVLink: NVIDIA's direct GPU-to-GPU interconnect inside a server.
- NVLink Switch: A switch chip that connects many NVLinks so all GPUs can talk to each other at full NVLink speed.
- Spectrum-X Ethernet: NVIDIA's AI-optimized Ethernet platform of switches and SuperNICs.
Try it
Sample question
What is NVLink, and how fast is it on a DGX H100?
Show the answer
Answer: A direct GPU-to-GPU interconnect inside a server. Fourth-generation NVLink on H100 gives 900 GB/s of bandwidth per GPU.
NVLink is NVIDIA's direct GPU-to-GPU interconnect. It scales multi-GPU input and output within a server. NVIDIA lists 1.5x higher bandwidth per GPU at 900 GBps with fourth-generation NVLink on DGX H100. Newer generations are faster, so always check which generation a question means.
What NVIDIA says (2)
“is a direct GPU-to-GPU interconnect that scales multi-GPU input/output (IO) in the server.”
“1.5X higher bandwidth per GPU @ 900 GBps with fourth generation of NVIDIA NVLink.”
Practice 2.9 (4 questions) Full AI Infrastructure guide
← 2.8 Data center networking protocols · 2.10 DPUs in the data center →