Aegis-GPU2.4 On-prem vs cloud
How to weigh up-front and ongoing costs, ownership and managed services.
Key points
TCO (total cost of ownership) is the full cost over the life of the system, including storage, compute and maintenance. ROI (return on investment) compares what you gain with what you spend. NVIDIA says IT leaders should evaluate TCO over time and treat ROI, not the initial TCO, as a key metric.
What NVIDIA says (2)
“IT leaders should evaluate the total cost of ownership (TCO) over time and consider factors such as data storage, compute resources, and ongoing maintenance.”
“In general, it’s important to consider return on investment (ROI) as a key metric, rather than the initial TCO.”
On-premises (on-prem) means the hardware sits in a data center you control. NVIDIA says DGX SuperPOD can be deployed on-premises, meaning the customer owns and manages the hardware as a traditional system. The trade-off is that you also own power, cooling, space and operations.
What NVIDIA says (1)
“DGX SuperPOD can be deployed on-premises, meaning the customer owns and manages the hardware as a traditional system.”
A fully managed service means the provider runs the infrastructure and you use it. NVIDIA describes DGX Cloud with Microsoft Azure, AWS, Google Cloud and OCI as a high-performance, fully managed AI training platform. An on-prem DGX SuperPOD, by contrast, is owned and managed by the customer.
What NVIDIA says (2)
“NVIDIA DGX Cloud with Microsoft Azure is a high-performance, fully managed AI training platform”
“DGX Cloud is built on NVIDIA-accelerated infrastructure and runs across CSPs and NVIDIA Cloud Partners (NCPs).”
CapEx (capital expenditure) is money spent up front to buy assets such as servers. OpEx (operational expenditure) is ongoing spending, such as a monthly cloud bill. NVIDIA says cloud reduces acquisition costs and shifts CapEx to OpEx. It also warns that long-term cloud expenses can add up.
What NVIDIA says (2)
“Cloud-based solutions offer a cost-effective way to start AI initiatives by reducing acquisition costs and shifting capital expenditures (CapEx) to operational expenditures (OpEx).”
“Yet, while cloud solutions may have lower initial costs, long-term expenses can add up.”
Key terms
- CapEx and OpEx: CapEx is money spent up front on assets; OpEx is ongoing spending such as a cloud bill.
- Total cost of ownership: The full cost of a system over its life, including compute, storage and maintenance.
Try it
Sample question
Your GPUs will run busy for several years. How does NVIDIA say to compare cloud with on-premises infrastructure?
Show the answer
Answer: Evaluate total cost of ownership (TCO) over time, and treat return on investment (ROI) as a key metric.
TCO (total cost of ownership) is the full cost over the life of the system, including storage, compute and maintenance. ROI (return on investment) compares what you gain with what you spend. NVIDIA says IT leaders should evaluate TCO over time and treat ROI, not the initial TCO, as a key metric.
What NVIDIA says (2)
“IT leaders should evaluate the total cost of ownership (TCO) over time and consider factors such as data storage, compute resources, and ongoing maintenance.”
“In general, it’s important to consider return on investment (ROI) as a key metric, rather than the initial TCO.”
Practice 2.4 (4 questions) Full AI Infrastructure guide
← 2.3 Power and cooling · 2.5 Components of an accelerated cluster →