7.1 Time series splits and forecast evaluation

Key points

1. Cross-validation means testing a model on several held-out slices of data. With time series, random shuffling lets the model peek at the future.

   What NVIDIA says (2)

   “Great care must be taken when defining cross-validation folds for time-series data.”

   — RAPIDS Deployment: Time series forecasting with HPO

   “We are not allowed to use the future to predict the past, so the training set must precede (in time) the validation set.”

   — RAPIDS Deployment: Time series forecasting with HPO

2. A forecast horizon is how many future steps you predict. Direct forecasting needs many models, so it costs more compute.

   What NVIDIA says (2)

   “Multistep forecasting One popular technique used in time series forecasting is recursive multi-step forecasting , in which you train a single model and apply it recursively to predict the next n values in the series.”

   — Accelerating Time Series Forecasting with RAPIDS cuML

   “In contrast, direct multi-step forecasting uses a separate model to predict each future value in your forecast horizon.”

   — Accelerating Time Series Forecasting with RAPIDS cuML

3. A lag feature is a past value of the target, such as last week's sales. A rolling-window statistic is a summary, such as a mean, over a recent time span.

   What NVIDIA says (2)

   “Lag features are useful because what happens in the past often influences what would happen in the future.”

   — RAPIDS Deployment: Time series forecasting with HPO

   “Rolling window statistics are statistics (e.g. mean, standard deviation) over a time duration in the past.”

   — RAPIDS Deployment: Time series forecasting with HPO

Key terms

• Rolling mean: The average over a sliding window of recent rows, used to smooth noise.
• Time series: Data points recorded in time order.
• Forecast horizon: How many future steps a forecast predicts.
• Lag feature: A past value of the target used as an input, such as last week's sales.

Try it

• Split lab: k-fold versus time-series validation

Sample question

How should you build cross-validation folds for a sales forecasting model?