6.1 Monitoring and optimizing ML pipelines

Key points

1. Monitoring means tracking a deployed model's behavior to analyze its performance. Software logic is fixed in code; a model's quality depends on the data it sees.

   What NVIDIA says (2)

   “Monitoring a machine learning model after deployment is vital, as models can break and degrade in production.”

   — A Guide to Monitoring Machine Learning Models in Production

   “Monitoring is not a one-time action that you do and forget about.”

   — A Guide to Monitoring Machine Learning Models in Production

2. Operational metrics show whether the service is healthy and fast. They complement model-quality metrics. I/O means input/output.

   What NVIDIA says (1)

   “Examples include: Latency IO/memory/disk use System reliability (uptime) Auditability”

   — A Guide to Monitoring Machine Learning Models in Production

3. Reliable pipelines need retries, run history and clear failure reports. An orchestrator provides them. MLOps means machine learning operations.

   What NVIDIA says (1)

   “Orchestration: Prefect flows chain the stages together, track runs, and handle failures”

   — RAPIDS Deployment: Fraud detection model lifecycle with cuDF, Prefect, MLflow and Triton

Key terms

• Prefect: A workflow orchestrator that chains pipeline stages, tracks runs, schedules them and handles failures.
• MLOps: Practices for building, deploying and running machine learning reliably in production.
• Model monitoring: Tracking a deployed model to catch slowdowns and drops in quality.

Sample question

Why is monitoring a deployed model different from monitoring traditional software?