Demystifying Weighted Mean Absolute Percentage Error for Data Scientists

As data scientists, evaluating the accuracy of our models is a critical part of the machine learning workflow. We need to understand how well our models are performing on real-world data in order to determine if they are ready for production use or if more tweaking and tuning is required. One popular technique for evaluating forecast accuracy is the weighted mean absolute percentage error, commonly referred to as WMAPE or sometimes wMAPE. But what exactly does this cryptic acronym mean and why is it useful? In this comprehensive guide, I’ll demystify WMAPE and explain when and how data scientists should use it.

Understanding Mean Absolute Percentage Error (MAPE)

First, to understand WMAPE, we need to understand the metric it is based on – mean absolute percentage error (MAPE) MAPE is used to measure the accuracy of forecasts or predictions compared to actual observed values It calculates the absolute percentage error for each data point, sums these errors, and divides by the number of fitted points. The formula looks like this

MAPE = (100/n) * Σnt=1 |(At – Ft)/At|

Where:

• n is the number of fitted data points
• At is the actual value
• Ft is the forecast value

MAPE provides an intuitive way to interpret model accuracy since the result is expressed as a percentage. A MAPE of 5% indicates the average forecast is off by 5% compared to the actuals. Easy to understand!

However, MAPE has some limitations:

• It breaks down when actual values are zero or close to zero, leading to divide by zero errors or extremely high MAPE values.
• It is biased – negative errors (over-forecasting) increase MAPE more slowly than positive errors (under-forecasting).
• All data points are weighted equally, even if some points are more important than others.

Introducing Weighted MAPE (WMAPE)

This brings us to weighted MAPE. WMAPE aims to overcome some of the limitations of regular MAPE by allowing users to assign weights to data points. The general formula is:

WMAPE = Σ (wi * |(Ai – Fi)/Ai|) / Σ wi

Where:

• wi is the weight assigned to the ith data point

By applying weights, we can give certain data points more influence over the final WMAPE value. For example, we may weight recent sales data higher than older data to better optimize for current accuracy. We can also downweight or even remove outliers that would otherwise skew results.

Typically, the weights used in WMAPE are based on the actual values themselves. For sales forecasting, using the sales volume as the weight is very common:

WMAPE = Σ (|Ai| * |(Ai – Fi)/Ai|) / Σ |Ai|

This helps minimize the impact of periods with very low actual values, since those points are automatically assigned lower weights.

When to Use WMAPE

Now that we understand how WMAPE works, when should we use it over regular MAPE?

Use WMAPE for intermittent or highly variable time series. For example, demand forecasting for retail products often fits this criteria. Sales can be bursty or seasonal with periods of low sales volume.

Use WMAPE when all data points are not equally important. For example, we may want to evaluate forecast accuracy just for the top 10 customers who make up 80% of sales. WMAPE allows us to weight those key accounts higher.

Use WMAPE to minimize the impact of outliers. Assigning lower weights to outliers prevents them from skewing the error metric as much.

Use WMAPE to optimize for recent accuracy. For models that should adapt quickly to changes, weighting recent data higher emphasizes short-term accuracy.

On the other hand, MAPE may be preferred over WMAPE if the actual values do not vary widely and all data points are equally important. WMAPE also introduces some complexity in choosing the right weights.

Example of Calculating WMAPE

Let’s walk through a concrete example to understand how to calculate WMAPE:

Actual | Forecast | Abs Error | Weight | Weighted Abs Error