The metric you optimise decides which forecast errors you stop caring about.

What each metric actually computes

MAE is the average absolute error in units. RMSE squares the errors before averaging and taking the root, so large misses count disproportionately. MAPE divides each error by its actual before averaging, producing a percentage. WAPE sums all absolute errors and divides by total actuals, producing a volume-weighted percentage. The first two are scale-dependent; the second two are not.

The distinction between MAPE and WAPE is the one that changes conclusions most often, and the arithmetic is worth being precise about. MAPE gives every item equal weight regardless of size: a two-unit miss on a three-unit item contributes 67%, the same weight as a 67% miss on a product selling ten thousand. WAPE pools the errors first, so items contribute in proportion to their volume.

There is also a property of RMSE and MAE that is easy to miss and decides which point of the distribution you are forecasting. Minimising MAE produces forecasts of the median; minimising RMSE produces forecasts of the mean. For a skewed demand distribution those are different numbers, and a model tuned on one will be systematically biased when judged on the other.

Why MAPE misleads on low-volume items

MAPE divides by the actual, so as the actual approaches zero the percentage error explodes, and at exactly zero it is undefined. Any catalogue with intermittent demand — spare parts, long-tail SKUs, new products, anything seasonal out of season — will therefore contain items whose MAPE is either enormous or uncomputable, and the usual response is to drop them from the calculation.

Dropping them is where the damage happens, because the excluded items do not stop existing. The reported figure now describes a subset of the catalogue chosen by the metric's own limitation, and it is reported as though it described the catalogue. Every stockout on an intermittent item is invisible to the number the planning team is measured on.

The standard forecasting reference is direct about this. Percentage errors are "infinite or undefined" when the actual is zero and take extreme values as it approaches zero; they assume a meaningful zero on the measurement scale; and they penalise negative errors more heavily than positive ones. That last asymmetry is the subtle one — under-forecasting is capped at 100% error while over-forecasting is unbounded, so a model tuned on MAPE is pushed toward under-forecasting.

For a demand planner that bias points the wrong way. Under-forecasting causes stockouts and lost sales; over-forecasting causes holding cost. A metric with a built-in preference for under-forecasting is quietly optimising toward the more expensive error in most retail and distribution contexts.

WAPE and what volume weighting hides

WAPE fixes MAPE's arithmetic problems: no division by individual actuals, so zeros are harmless, and error weighted by volume, so the items that matter commercially dominate the number. For a portfolio-level view of whether forecasting is working, it is usually the better default — which is why it is the standard in retail and supply chain planning.

The trade-off is that weighting by volume means small items become invisible. A catalogue where the top fifty products are forecast well and the remaining three thousand are forecast badly can post an excellent WAPE, and that is arithmetically correct — those three thousand items are a small share of volume. Whether it is commercially correct depends on whether the tail carries margin, strategic range or service-level commitments.

The practical answer is to report WAPE at portfolio level and segment beneath it, instead of choosing between the two. WAPE overall, WAPE by category, and a separate service-level measure for the tail gives a planner somewhere to look when the headline is fine and the phone is ringing.

• WAPE for portfolio-level reporting and period-on-period comparison
• Segment beneath it, because a good aggregate can hide a badly forecast tail
• Never compare WAPE across businesses — it depends on the demand profile, not just the model
• Report the denominator alongside it; WAPE on a shrinking base rises without the forecast changing

RMSE, outliers and when the squaring is the point

RMSE squares each error before averaging, so a single large miss affects it far more than several small ones. Whether that is a feature or a distortion depends entirely on whether large errors are disproportionately costly in the actual business — and often they are, because one large stockout is worse than a scattering of small ones.

The failure to watch for is a promotional spike or a data error dominating the metric. One anomalous week can move RMSE enough to make a genuinely improved model look worse, and teams then tune against an outlier rather than against demand. Inspecting the largest contributing errors before acting on an RMSE movement is a five-minute check that prevents a fortnight of wrong work.

RMSE is also scale-dependent, which makes it useful within one series and meaningless across a catalogue of different sizes. Comparing RMSE for a product selling ten units a week with one selling ten thousand tells you about the sizes of the products, not about the quality of the forecasts.

Bias is the measure everyone forgets

Every metric above uses absolute or squared error, which means all of them are blind to direction. A forecast that is 10% high half the time and 10% low the other half scores identically to one that is 10% high every single period — and those are completely different problems. Mean error, tracked as a signed value, is the only one that catches persistent bias.

Persistent bias is the most actionable forecast defect there is, because it is usually systematic and therefore fixable: a promotional uplift not modelled, a discontinued line still forecast, a calendar effect misaligned, a planner override applied in one direction. Accuracy metrics will show it as ordinary error and give no clue where to look.

Track signed bias alongside whichever accuracy metric you choose, at the same level of aggregation, and watch its sign over consecutive periods instead of its magnitude in any one. A bias that alternates is noise. A bias with the same sign for six periods running is a model or process problem waiting to be found.

Aggregation changes the answer

Forecast error shrinks as you aggregate, because errors in opposite directions offset. The same forecasts can show 30% error at SKU-store level, 15% at SKU-region and 6% at national. None of those is the true number; they answer different questions, and quoting one without its level is close to meaningless.

The level that matters is the one at which decisions are taken. Replenishment happens at SKU-store, so a national accuracy figure says nothing about whether shelves will be full. Capacity planning happens at network level, so SKU-store error is the wrong lens. Reporting accuracy at a level nobody acts on is how a planning function reports improvement while service levels decline.

This is also the most common way forecast accuracy gets unintentionally flattered. Aggregating up is a legitimate analytical step and an easy accidental misrepresentation, and the defence is simply to state the level beside every number.

A worked example, with hypothetical numbers

The numbers below are invented to make the arithmetic visible — they are not measured results and describe no real business. Two products over one period: Product A forecast 1,000 against 1,100 actual; Product B forecast 8 against 2 actual. The metrics disagree sharply about how good this forecast was.

MAPE averages the two percentage errors: Product A is off by 100 on 1,100, about 9.1%; Product B is off by 6 on 2, or 300%. The mean is roughly 155%, which reads as a catastrophic forecast. WAPE pools first: total absolute error 106 against total actual 1,102, giving about 9.6% — which reads as a good forecast. The same two rows, two conclusions.

Both numbers are arithmetically right and they answer different questions. If the business cares about units and revenue, 9.6% is the honest summary and Product B is a rounding error. If Product B is a critical spare whose absence halts a production line, the 300% matters far more than the 9.1%, and neither aggregate captures that — a service-level measure does.

Extend it one step: add a third product forecast at 5 against an actual of 0. MAPE is now undefined and the row is silently dropped from most implementations, after which the reported figure covers two of three products while claiming to describe the catalogue.

Choosing by what the error costs

The selection rule that survives contact with a real planning process is to pick the metric whose blind spot you can most afford. Every one of these measures ignores something; the question is whether the thing it ignores is the thing that hurts. Cost asymmetry, demand intermittency and decision level determine the answer, not statistical elegance.

Read the limitation column as the operative one — it is what the metric will stop the team from seeing, in exchange for the clarity it provides.

How a metric gets gamed without anyone cheating

Any forecast metric used as a target will be optimised, including in ways nobody intended. This does not require bad faith — it requires a team doing exactly what they were asked. The common patterns are worth recognising because they all produce an improving number and a deteriorating business.

Excluding hard items is the most frequent: new products, promotions and intermittent lines get removed from the accuracy calculation because they are volatile, and the remaining figure improves every quarter while the excluded set grows. Reporting the coverage percentage alongside the accuracy figure closes this immediately.

Reporting at a higher aggregation improves the number without improving any decision. Forecasting conservatively low improves MAPE specifically, because of its asymmetric penalty, while causing exactly the stockouts the forecast exists to prevent. And smoothing a forecast toward the mean improves RMSE while removing the peaks that drive capacity planning.

The defence is not a better single metric. It is reporting a small fixed set — one accuracy measure, signed bias, and item coverage, all at the decision level — and leaving the definition alone for long enough that changes in the number mean something.

What none of these will tell you

Accuracy is a proxy. The business outcome is service level, working capital and margin, and a more accurate forecast improves those only if someone changes a decision because of it. A planning process where forecasts improve and order quantities are still set by a rule of thumb will show better metrics and identical results.

It is also worth stating that some demand is not forecastable to any useful precision — genuinely intermittent, externally driven, or dominated by one-off events. For those, the honest response is to stop trying to improve the point forecast and manage the uncertainty instead, through safety stock, flexible supply or a range instead of a number. Turning a prediction into a decision is where that work belongs, and it is a different discipline from improving the model.

Two further limits. None of these metrics accounts for forecast horizon, and a one-week-ahead figure is not comparable to a thirteen-week-ahead one. And none of them is meaningful unless the actuals are reliable: an accuracy programme on a demand history that does not distinguish true demand from censored demand during a stockout is measuring the wrong series, which is a data quality problem before it is a modelling one.

Questions and trade-offs

Technical references

• Hyndman & Athanasopoulos: evaluating point forecast accuracy ↗
• scikit-learn: metrics and scoring for regression ↗