Aegis-GPU1.6 Installing GPU servers
Who installs a DGX, how it sits in the rack, first boot and reading the GPU-NIC topology.
Key points
nvidia-smi (System Management Interface) is the standard command-line tool for NVIDIA GPUs. 'nvidia-smi topo -m' prints a matrix of the links between all GPUs and NICs, plus the CPU and memory affinity of each GPU. Affinity tells you which CPU socket and memory are closest to a GPU.
What NVIDIA says (1)
“Topology connections and affinities matrix between the GPUs and NICs in the system nvidia-smi topo -m”
A warranty covers repair costs. NVIDIA ties it to an approved installer. The installer also performs first boot setup. The steps are documented so you can re-image the system later.
What NVIDIA says (2)
“Your DGX H100/H200 system must be installed by NVIDIA partner network personnel or NVIDIA field service engineers.”
“If not performed accordingly, your hardware warranty will be voided.”
A rack mount kit fixes a server into the rack posts. The DGX H100 kit is a shelf, not sliding rails. The system stays put. You service parts from the front or rear.
What NVIDIA says (1)
“The rack mount kit acts as a shelf in the rack, it does not allow the system to be moved once installed. All components are serviceable from the front or rear.”
First boot setup runs the first time the system powers on after delivery or re-imaging. It creates one admin account for the OS, the BMC and the GRUB boot loader, and sets up the primary network interface.
What NVIDIA says (2)
“Create an administrative user account for the system, BMC, and Grub boot loader.”
“Configure the primary network interface.”
Key terms
- First boot setup: The wizard that runs the first time a DGX starts, creating the admin account and the primary network setup.
Sample question
You install a GPU server and want to see how its GPUs and NICs connect to each other and to CPUs. Which nvidia-smi command shows this?
Show the answer
Answer: nvidia-smi topo -m
nvidia-smi (System Management Interface) is the standard command-line tool for NVIDIA GPUs. 'nvidia-smi topo -m' prints a matrix of the links between all GPUs and NICs, plus the CPU and memory affinity of each GPU. Affinity tells you which CPU socket and memory are closest to a GPU.
What NVIDIA says (1)
“Topology connections and affinities matrix between the GPUs and NICs in the system nvidia-smi topo -m”
Practice 1.6 (4 questions) Full System and Server Bring-up guide
← 1.5 Power and cooling · 1.7 Validating installed hardware →