STOP 10 / 31What did your model leave behind?
Layout
STOP 10 · Signal

What did your model leave behind?

EXPLORE~6 min
Back to route map
  • A residual is an observed value minus its fitted one-step prediction.
  • If the remaining errors still have a weekly rhythm, Northstar’s model has left useful information on the table.
White noise & residuals: concepts and Northstar examples. Illustrative Northstar examples, not measured business results.
ConceptWhat it meansNorthstar example
Look for zero mean and no autocorrelation.
  • These are key desirable residual properties.
  • Roughly constant variance helps with uncertainty estimation.
  • Normality is convenient for some interval formulas, not a universal requirement for useful forecasts.

If Northstar's model is systematically too low on Mondays, a useful weekly signal remains in its residuals.

Uncorrelated does not mean independent.
  • White noise has zero mean, constant finite variance, and no serial correlation.
  • Nonlinear dependence can still remain.

No linear lag correlation does not prove that Northstar's remaining errors are independent; changing volatility can still carry structure.

Diagnostics don’t replace validation.
  • Use residual plots, ACF, and a suitable Ljung–Box test.
  • Passing a diagnostic does not prove accuracy; still evaluate future holdout data.
  • A model can pass a residual check but lose to seasonal naïve on the next Black Friday.
  • Diagnostics and out-of-sample testing answer different questions.
FIG 10.1

Is there a pattern left over?

WHAT THIS SHOWS

Fit a simple model to Northstar's first 224 simulated days, then inspect its errors on the next 96 unseen days. Compare the time plot, correlation bars, average miss, and signed bias together.

The question it answers: Has the model missed a predictable pattern, and are its errors centered near zero?

Model

Held-out errors · actual − forecast

Held-out Northstar forecast errors by day-20.0-5.59.023.6First held-out dayDay 96
ErrorZero

ACF of the errors

-1-0.500.5117142128Lag in days · red marks multiples of 7
Ljung–Box Q · 14 lags489.7
Average miss · MAE7.60
Bias · actual − forecast1.10

Correlation remains at the approximate 14-lag reference: inspect the omitted structure.

Fine print & method

Models are ordinary least squares fitted only to training data: intercept, time, and optionally six weekday indicators. The biased option adds a supplied +5 offset to the same fitted forecast. ACF uses centered sample autocovariances. Q is computed from 14 lags; 23.685 is a rough χ²(14) reference, not a calibrated production verdict for estimated models. Passing it does not prove independence, normality, zero bias, or good forecasts.

CHECK YOURSELF · CHOOSE AN ANSWER

A residual ACF has a repeated peak at lag 7. What should you investigate?

Your learning note

Private · just for you