- You have a sales dataset and someone asks for a forecast.
- Where do you begin?
- Not with ARIMA orders or a neural network.
- Start with the decision, understand the data, build a baseline, and let honest tests tell you what to try next.
- This is a reusable roadmap—not a ladder you must climb to the most complicated model.
| Concept | What it means | Northstar example |
|---|---|---|
| First define what a useful forecast means. |
|
|
| Explore before fitting; establish a baseline before tuning. |
| Inspect Northstar's missing days, stockouts, weekly rhythm, and Black Friday timing before choosing a simple event-aware baseline. |
| Design an honest test, then diagnose the misses. |
| Test future periods from several historical origins, then ask whether Northstar's misses are bias, omitted events, missing seasons, or residual dependence. |
| Add the component that the evidence calls for. |
|
|
| Keep complexity only when it earns its place. |
|
|
Your roadmap for a new forecasting problem
Open one stage at a time and map a new Northstar forecasting problem to a decision, honest test, and targeted candidate.
The question it answers: What evidence should determine the next step—not a mandatory complexity ladder?
Select a step to read its explanation right underneath. Only one step stays open at a time.
- WHAT TO DO
Write a one-paragraph forecasting brief before touching a model: what quantity, for whom, at what granularity, how far ahead, and what action depends on it? Decide whether you need individual days, the horizon total, or both.
AT NORTHSTARNorthstar: daily net sales by category for the next 42 days, refreshed weekly. The inventory team also needs plausible high-demand scenarios before Black Friday. Clarify how sales, prices, and availability relate to the units they must stock.
LEAVE THIS STEP WITHAn agreed target, information cut-off, horizon, metric, and business owner.
- WHAT TO DO
Check dates, units, missing observations versus true zeros, stockouts, returns, outliers, and changes in the business. Plot training history, seasonal views, and ACF. Consider log/Box–Cox transforms when the scale warrants it; fit all preprocessing inside each training window.
AT NORTHSTARNorthstar has weekly shopping rhythms, growing sales, and an annual event whose date moves. STL/MSTL can help inspect components, but an event-relative Black Friday view exposes something a fixed weekday or date cannot.
LEAVE THIS STEP WITHAn annotated series and a short list of plausible patterns—not a declaration of the winning model.
- WHAT TO DO
Try the last observation, a seasonal naïve forecast, and another domain-appropriate simple method. Use the same weekday last week for a weekly rhythm. Last-year comparisons require enough history and careful calendar alignment; estimate growth using training data only.
AT NORTHSTARNorthstar benchmarks last-week sales and an event-aligned last-year forecast with a training-only growth estimate. A blind 364-day copy aligns weekdays but can put Black Friday on the wrong Friday.
LEAVE THIS STEP WITHA reproducible benchmark, plus a fallback if a more complex forecasting system fails.
- WHAT TO DO
Define the evaluation protocol before tuning: rolling origins, deployment frequency, forecast horizon, and decision-relevant metrics. Avoid random splits. Refit models and preprocessing at each origin, with only information available then. Where history allows, reserve a final chronological test period for one last check after selection.
AT NORTHSTARNorthstar makes 42-day forecasts from several earlier weekly origins. Include origins before Black Friday and ordinary periods. With only a few annual events, report that holiday performance is estimated from limited evidence.
LEAVE THIS STEP WITHComparable validation forecasts, MAE or other chosen loss, bias, interval coverage, and event-specific errors.
- WHAT TO DO
Look at held-out errors by horizon, weekday, product, and event. Use residual plots, ACF, and suitable diagnostics as clues. Is the miss a shifted level, trend, seasonal pattern, calendar effect, autocorrelated error, or a data-quality problem? Check inputs before adding parameters.
AT NORTHSTARNorthstar’s weekly model does fine on ordinary days but misses the Black Friday lift. That points to a missing calendar effect. Increasing trend flexibility would solve the wrong problem.
LEAVE THIS STEP WITHA testable hypothesis: ‘Adding an event window should reduce holiday underprediction without worsening ordinary days.’
- WHAT TO DO
Fit a small number of plausible candidates. Use the model guide below to match a missing pattern to a method. Compare Prophet, DHR, and MSTL + ETS where appropriate. Fourier terms are seasonal regression features, not a required step after ETS. ARIMA can be a standalone alternative or model errors left after regression.
AT NORTHSTARFor Northstar, a candidate might combine a trend, weekday effects, smooth annual Fourier terms, and explicit Black Friday/Cyber Monday windows. Add ARIMA errors only if serial dependence remains and validation supports the change.
LEAVE THIS STEP WITHOne clearly motivated experiment with known-ahead features and a versioned set of assumptions.
- WHAT TO DO
Compare every candidate against the baseline on identical origins and horizons. Inspect the size and consistency of improvements, interval calibration, important segments, runtime, and maintenance. Repeated tuning can overfit validation too; confirm the selected approach on the reserved test period where feasible.
AT NORTHSTARNorthstar keeps the event-aware approach only if the holiday improvement is credible and ordinary-day performance remains acceptable. If no candidate adds reliable value, the baseline can be the production choice. Combining good candidates is another experiment, not a guaranteed win.
LEAVE THIS STEP WITHA selected method—or a documented reason to return to the audit/diagnosis stage. A simple model is allowed to win.
- WHAT TO DO
Refit the chosen specification on information available at the current origin. Publish point forecasts, calibrated intervals where supported, scenarios, and assumptions. Reconcile totals when needed. Automate data checks, refreshes, fallbacks, and monitoring of error, bias, coverage, and drift.
AT NORTHSTARNorthstar checks weekly sales bias and interval coverage, then separately reviews Black Friday outcomes. A new pricing policy or persistent demand shift sends the team back to diagnosis, not automatically to a more complicated algorithm.
LEAVE THIS STEP WITHA usable forecast, a responsible owner, a refresh cadence, and a trigger for revisiting the method.
No reliable improvement? Keep the simpler method, or return to audit and diagnosis. You do not have to try every model family.
What should I try next?
Pick the pattern you observed. This suggests an experiment—not a guaranteed winner.
- KEEP AS THE BENCHMARK
- Weekly naïve plus a separate event-aligned last-year benchmark
- CANDIDATE TO TEST
- Compare calendar/Fourier regression, Prophet, and MSTL + ETS with event handling
Weekday and annual Fourier features plus event-relative windows form one candidate; DHR adds ARIMA errors if useful. Prophet provides a trend/seasonality/holiday alternative. MSTL + ETS separates multiple seasons but needs deliberate event adjustment and add-back. Compare their complete pipelines under the same validation protocol.
Watch out: Neither smooth Fourier waves nor a fixed seasonal slot reliably aligns a sharp moving holiday by itself. Tune within time-aware validation, and use scenarios for unknown future drivers.
Agree on the 42-day decision → audit sales and availability → benchmark weekly and event-aligned naïve forecasts → evaluate rolling 42-day forecasts → inspect weekday and Black Friday misses → test calendar/Fourier regression with explicit event windows → add ARIMA errors only if useful → compare, refit, and monitor.
STL/MSTL helps explore components. ETS is a forecasting family. Fourier terms represent smooth seasonality. They solve different problems and are not mandatory stages in one fixed sequence.
Fine print & assumptions
Decision guide: validate each candidate on the actual business decision and future-data tests.