1.10 Kubernetes with BCM

Key points

1. A TUI is a text-based menu interface in the terminal. cm-kubernetes-setup also has a command-line mode for automated installs.

   What NVIDIA says (1)

   “The usual, and recommended way, to install Kubernetes with BCM is to install it interactively from a TUI session”

   — NVIDIA Base Command Manager 11 Containerization Manual (PDF)

2. The GPU Operator installs the device plugin that advertises GPUs to Kubernetes. Without it, no node offers GPUs, so GPU pods cannot be scheduled.

   What NVIDIA says (2)

   “The NVIDIA GPU Operator is not selected by default.”

   — NVIDIA Base Command Manager 11 Containerization Manual (PDF)

   “A cluster with NVIDIA GPUs therefore cannot run GPU workloads until the operator is installed.”

   — NVIDIA Base Command Manager 11 Containerization Manual (PDF)

3. etcd is the key-value store that holds Kubernetes state. It needs a majority to work, so an odd count avoids ties. Three nodes avoid a single point of failure.

   What NVIDIA says (2)

   “An etcd cluster—the Kubernetes distributed key-value storage—runs on an odd number (1, 3, 5 ...) of nodes.”

   — NVIDIA Base Command Manager 11 Containerization Manual (PDF)

   “a minimum of three nodes is recommended for etcd”

   — NVIDIA Base Command Manager 11 Containerization Manual (PDF)

Key terms

• etcd: The distributed key-value store that holds Kubernetes cluster state; it runs on an odd number of nodes.
• NVIDIA GPU Operator: A Kubernetes operator that installs and manages the GPU driver, container toolkit, device plugin and related parts.

Sample question

Which BCM tool installs Kubernetes, and how does NVIDIA recommend running it?