2.8 Energy-efficient, trustworthy models

Key points

1. Quantization stores and computes with fewer bits, such as INT8, to save time and energy. NVIDIA uses ModelOpt to calibrate and quantize the SDXL UNet. VAE means variational autoencoder. INT8 means 8-bit integer.

   What NVIDIA says (1)

   “The UNet part typically consumes >95% of the e2e Stable Diffusion latency.”

   — NeMo Framework 24.09: Stable Diffusion XL Int8 Quantization

2. Less data means fewer training steps and less energy. Removing near-duplicates keeps most of the useful signal.

   What NVIDIA says (1)

   “it can remove up to 50% of the data with minimal performance loss.”

   — NeMo Curator (NeMo Framework 24.09): Semantic Deduplication

3. Trustworthy AI puts safety and transparency first. Being transparent includes sharing benchmarks and dataset descriptions.

   What NVIDIA says (1)

   “They’re also transparent — providing information such as accuracy benchmarks or a description of the training dataset”

   — What Is Trustworthy AI?

Key terms

• Semantic deduplication: Removing data pairs that mean nearly the same thing, found by clustering embeddings and comparing cosine similarity.
• Quantization: Running a model with fewer bits, such as INT8, to cut latency and energy.
• Trustworthy AI: An approach to AI development that puts safety and transparency first.

Sample question

To make Stable Diffusion XL cheaper to run, which part should you quantize first, and why?