5.2 Identifying faulty parts
Finding a failed PSU, GPU or card from LEDs, NVSM and run-to-run differences.
Key points
A PSU (power supply unit) feeds the system. The DGX H100 names them PSU0 to PSU5. A failed one shows an amber LED. nvsm show psus reports each PSU's health.
What NVIDIA says (2)
“Identify the broken power supply either by the amber color LED or by the power supply number”
“Look for any that do not report Status_Health=OK .”
A baseline is a known-good result to compare against. NVIDIA expects the same test on different parts of the system to perform similarly. A repeatable gap on the same hardware points to a component in that hardware.
What NVIDIA says (2)
“When running the following tests, you should expect that performance between runs of the same configuration on distinct parts of the system should run in a similar time or at a similar performance level.”
“However, over multiple runs on the same sets of hardware a difference is found, it can indicate an issue with some component of that system.”
Sample question
How can you identify a failed power supply on a DGX H100?
Show the answer
Answer: Look for an amber LED, run sudo nvsm show psus and find any PSU without Status_Health=OK, or check BMC sensors.
A PSU (power supply unit) feeds the system. The DGX H100 names them PSU0 to PSU5. A failed one shows an amber LED. nvsm show psus reports each PSU's health.
What NVIDIA says (2)
“Identify the broken power supply either by the amber color LED or by the power supply number”
“Look for any that do not report Status_Health=OK .”
Practice 5.2 (2 questions) Full Troubleshoot and Optimize guide