3.2 Installing the OS on nodes

NCP-AII · Control Plane Installation and Configuration (19% of the exam) · Official objective: “Install OS.”

PXE boot and how BCM provisioning nodes push software images.

Key points

Provisioning means BCM sends a full software image to a node over the network. For that to work, the node must PXE boot, which means boot from the network. So the guide sets Boot Option #1 to [NETWORK] in the DGX BIOS.
What NVIDIA says (2)
“Configure the DGX systems to PXE boot by default.”
— DGX SuperPOD Deployment Guide: Initial Cluster Setup
“enter the BIOS menu, and configure Boot Option #1 to be [NETWORK]”
— DGX SuperPOD Deployment Guide: Initial Cluster Setup
A software image is the full OS file tree that a node runs. Provisioning copies that image to the node. A provisioning node is any node with the provisioning role. The head node always has it.
What NVIDIA says (2)
“The action of transferring the software image to the nodes is called node provisioning and is done by special nodes called the provisioning nodes.”
— DGX SuperPOD Administration Guide: Provisioning Nodes
“Creating provisioning nodes is done by assigning a provisioning role to a node or category of nodes.”
— DGX SuperPOD Administration Guide: Provisioning Nodes
Each provisioning node sends images to a limited number of nodes at the same time. That limit is provisioning slots. The default of 10 is safe for typical clusters. Setting it lower can prevent network and disk overload.
What NVIDIA says (2)
“The maximum number of nodes that can be provisioned in parallel by the provisioning node.”
— DGX SuperPOD Administration Guide: Provisioning Nodes
“The default value is 10, which is safe for typical cluster setups.”
— DGX SuperPOD Administration Guide: Provisioning Nodes

Key terms

System BIOS: The firmware that starts the server before the OS loads; it holds settings such as TPM and boot order.
Base Command Manager: NVIDIA cluster management software that provisions nodes, manages categories and installs Slurm.
PXE boot: Booting a node from the network so the provisioning node can send it a software image.

Sample question

BCM will install the OS image on your DGX nodes over the network. What BIOS change does the SuperPOD guide make on each DGX?

Show the answer

Answer: Set Boot Option #1 to [NETWORK] so the node PXE boots by default.

Provisioning means BCM sends a full software image to a node over the network. For that to work, the node must PXE boot, which means boot from the network. So the guide sets Boot Option #1 to [NETWORK] in the DGX BIOS.

What NVIDIA says (2)

“Configure the DGX systems to PXE boot by default.”
— DGX SuperPOD Deployment Guide: Initial Cluster Setup

“enter the BIOS menu, and configure Boot Option #1 to be [NETWORK]”
— DGX SuperPOD Deployment Guide: Initial Cluster Setup

Practice 3.2 (3 questions) Full Control Plane Installation and Configuration guide

← 3.1 BCM install and HA · 3.3 Cluster setup: categories, Slurm, Enroot, Pyxis →