- Northstar needs to know both typical error and the cost of major misses.
- No single score tells the whole story, especially when a holiday weekend matters far more than an ordinary Tuesday.
| Metric | What it means | Northstar example |
|---|---|---|
| MAE | Mean absolute error: average size of the miss in the original units. | Interpret Northstar's miss in dollars or index points, but do not compare raw MAE across businesses of very different sizes. |
| RMSE | Square errors, average them, then take the square root; large misses get more influence. |
|
| MAPE | Mean absolute error as a percentage of each actual observation. | Near-zero Northstar sales produce extreme percentages; an actual of zero makes the ordinary formula undefined. |
| WAPE | Sum of absolute errors divided by sum of absolute actuals. |
|
| MASE | Test MAE divided by a naïve error scale computed on training history. |
|
| Bias | Mean signed error, using actual minus forecast. |
|
One big miss changes the score
Compare Northstar's supplied monthly forecasts with simulated actual sales. Change systematic error, zero-mean noise, one large miss, and a zero-sales month to see which metrics react.
The question it answers: Do you miss by a lot, systematically in one direction, or mainly on a few extreme/low-sales months?
Fine print & method
These are supplied forecast perturbations, not fitted production predictions. Noise is centered to have zero average additive effect. MAPE is undefined at a zero actual; WAPE is finite here because aggregate actual sales remain positive, not because it always solves zero-denominator problems. MASE divides test MAE by a one-step naïve MAE computed from separate supplied training observations. Positive bias means underprediction.