6.2 Tracking experiments

Key points

1. Experiment tracking records what was run, with which settings, and how it scored. That lets you compare runs and reproduce the best one. ML means machine learning. XGBoost means eXtreme Gradient Boosting.

   What NVIDIA says (1)

   “Experiment tracking: MLflow logs hyperparameters, model artifacts, and evaluation metrics for every run”

   — RAPIDS Deployment: Fraud detection model lifecycle with cuDF, Prefect, MLflow and Triton

2. Weights & Biases (W&B) is an experiment tracking platform. Like MLflow, it records runs so you can debug and reproduce models. ML means machine learning.

   What NVIDIA says (1)

   “Weights & Biases : W&B’s tools help many ML users build better models faster, debugging and reproducing their models with just a few lines of code”

   — What is MLOps?

3. MLOps means machine learning operations: practices for running AI reliably in production. Tracking ties each model to its data, code and settings.

   What NVIDIA says (1)

   “AI models require careful tracking through cycles of experiments, tuning and retraining.”

   — What is MLOps?

Key terms

• MLflow: An open-source tool that records experiment settings, metrics and models, and keeps a model registry.
• Weights & Biases: A platform for tracking, debugging and reproducing ML experiments.
• MLOps: Practices for building, deploying and running machine learning reliably in production.
• Experiment tracking: Recording the settings, data and results of every training run so runs can be compared and repeated.
• Model artifact: A file produced by a run, such as a saved model, that is stored with the run record.

Sample question

What does MLflow do in NVIDIA's fraud detection pipeline?