The M5 competition: Background,organization, and implementation

The M5 competition follows the previous four M competitions, whose purpose is to learn from empirical evidence how to improve forecasting performance and advance the theory and practice of forecasting. M5 focused on a retail sales forecasting application with the objective to produce the most accurate point forecasts for 42,840 time series that represent the hierarchical unit sales of the largest retail company in the world, Walmart, as well as to provide the most accurate estimates of the uncertainty of these forecasts. Hence, the competition consisted of two parallel challenges, namely the Accuracy and Uncertainty forecasting competitions. M5 extended the results of the previous M competitions by: (a) significantly expanding the number of participating methods, especially those in the category of machine learning; (b) evaluating the performance of the uncertainty distribution along with point forecast accuracy; (c) including exogenous/explanatory variables in addition to the time series data; (d) using grouped, correlated time series; and (e) focusing on series that display intermittency. This paper describes the background, organization, and implementations of the competition, and it presents the data used and their characteristics. Consequently, it serves as introductory material to the results of the two forecasting challenges to facilitate their understanding.     

Retail forecasting: Research and practice

This paper reviews the research literature on forecasting retail demand. We begin by introducing the forecasting problems that retailers face, from the strategic to the operational, as sales are aggregated over products to stores and to the company overall. Aggregated forecasting supports strategic decisions on location. Product-level forecasts usually relate to operational decisions at the store level. The factors that influence demand, and in particular promotional information, add considerable complexity, so that forecasters potentially face the dimensionality problem of too many variables and too little data. The paper goes on to evaluate evidence on comparative forecasting accuracy. Although causal models outperform simple benchmarks, adequate evidence on machine learning methods has not yet accumulated. Methods for forecasting new products are examined separately, with little evidence being found on the effectiveness of the various approaches. The paper concludes by describing company forecasting practices, offering conclusions as to both research gaps and barriers to improved practice.     

Probabilistic forecasting of heterogeneous consumer transaction–sales time series

We present new Bayesian methodology for consumer sales forecasting. Focusing on the multi-step-ahead forecasting of daily sales of many supermarket items, we adapt dynamic count mixture models for forecasting individual customer transactions, and introduce novel dynamic binary cascade models for predicting counts of items per transaction. These transaction–sales models can incorporate time-varying trends, seasonality, price, promotion, random effects and other outlet-specific predictors for individual items. Sequential Bayesian analysis involves fast, parallel filtering on sets of decoupled items, and is adaptable across items that may exhibit widely-varying characteristics. A multi-scale approach enables information to be shared across items with related patterns over time in order to improve prediction, while maintaining the scalability to many items. A motivating case study in many-item, multi-period, multi-step-ahead supermarket sales forecasting provides examples that demonstrate an improved forecast accuracy on multiple metrics, and illustrates the benefits of full probabilistic models for forecast accuracy evaluation and comparison.     

Evaluating alternative models of trend inflation

With the concept of trend inflation now being widely understood to be important to the accuracy of longer-term inflation forecasts, this paper assesses alternative models of trend inflation. Reflecting the models which are common in reduced-form inflation modeling and forecasting, we specify a range of models of inflation that incorporate different trend specifications. We compare the models on the basis of their accuracies in out-of-sample forecasting, both point and density. Our results show that it is difficult to say that any one model of trend inflation is the best. Several different trend specifications seem to be about equally accurate, and the relative accuracy is somewhat prone to instabilities over time.     

Principles and algorithms for forecasting groups of time series: Locality and globality

Global methods that fit a single forecasting method to all time series in a set have recently shown surprising accuracy, even when forecasting large groups of heterogeneous time series. We provide the following contributions that help understand the potential and applicability of global methods and how they relate to traditional local methods that fit a separate forecasting method to each series:

• •Global and local methods can produce the same forecasts without any assumptions about similarity of the series in the set.
• •The complexity of local methods grows with the size of the set while it remains constant for global methods. This result supports the recent evidence and provides principles for the design of new algorithms.
• •In an extensive empirical study, we show that purposely naïve algorithms derived from these principles show outstanding accuracy. In particular, global linear models provide competitive accuracy with far fewer parameters than the simplest of local methods.

     

The Predictive Ability of Geographic Segment Information at the Country, Continent, and Consolidated Levels

This study examines whether geographic information disclosed at an increasingly disaggregated level (specifically, consolidated vs. continent vs. country) results in increased predictive ability of company operations (specifically, sales, gross profit, and earnings). Multinational corporations (MNCs) are formed using a simulated merger approach by combining the annual operating results of six individual firms, one from each of six countries. This approach makes it possible to compare the forecasting accuracy of data disclosed at the country, continent, and consolidated levels, not possible using current geographic segment disclosures. Previous studies using year-ahead forecast models implicitly assume the predictive factors included in the models are significant in forecasting operating results. Using regression forecast models, this study tests whether the predictive factors included in the models are effective in forecasting operating results by examining the direction, size, and significance of the regression coefficient estimates. The coefficients provide evidence that exchange rate changes, inflation, and real GNP growth are useful in forecasting annual sales and gross profit. Whereas, at least for this sample and this time period, exchange rate changes, inflation, and real GNP growth are not significant variables in forecasting annual earnings. The results indicate that the accuracy of forecasts increase as sales and gross profit are disclosed at a more disaggregated geographic level. The hypothesized relationship between consolidated, continent, and country levels, while holding strongly under perfect foresight, holds to a lesser extent using forecasts of exchange rates, inflation, and real GNP.     

Macroeconomic Forecasting Performance under Alternative Specifications of Time‐Varying Volatility

This paper compares alternative models of time‐varying volatility on the basis of the accuracy of real‐time point and density forecasts of key macroeconomic time series for the USA. We consider Bayesian autoregressive and vector autoregressive models that incorporate some form of time‐varying volatility, precisely random walk stochastic volatility, stochastic volatility following a stationary AR process, stochastic volatility coupled with fat tails, GARCH and mixture of innovation models. The results show that the AR and VAR specifications with conventional stochastic volatility dominate other volatility specifications, in terms of point forecasting to some degree and density forecasting to a greater degree. Copyright © 2014 John Wiley & Sons, Ltd.     

Exploring the representativeness of the M5 competition data

The main objective of the M5 competition, which focused on forecasting the hierarchical unit sales of Walmart, was to evaluate the accuracy and uncertainty of forecasting methods in the field to identify best practices and highlight their practical implications. However, can the findings of the M5 competition be generalized and exploited by retail firms to better support their decisions and operation? This depends on the extent to which M5 data is sufficiently similar to unit sales data of retailers operating in different regions selling different product types and considering different marketing strategies. To answer this question, we analyze the characteristics of the M5 time series and compare them with those of two grocery retailers, namely Corporación Favorita and a major Greek supermarket chain, using feature spaces. Our results suggest only minor discrepancies between the examined data sets, supporting the representativeness of the M5 data.     

Dynamic advertising and pricing with constant demand elasticities

In this paper we analyze a stochastic dynamic advertising and pricing model with isoelastic demand. The state space is discrete, time is continuous and the planing horizon is allowed to be finite or infinite. A dynamic version of the Dorfman–Steiner identity will be derived. Explicit expressions of the optimal advertising and pricing policies, of the value function and of the optimal advertising expenditures will be given. The general results will be used to analyze the case of impatient customers. Furthermore, particular time inhomogeneous models and homogeneous ones with and without discounting will be examined. We will study the social efficiency of a monopolist's optimal policies and the consequences of specific subsidies. From a buyer's perspective, our analysis reveals that waiting – when looking at (immediate) expected prices – is never profitable should two or more units be available. But we will also prove that the sequence of average sales prices is monotone decreasing. Moreover, the techniques applied to solve the discrete stochastic advertising and pricing problem will be used to solve a related deterministic control problem with continuous state space.     

Fast and accurate yearly time series forecasting with forecast combinations

It has long been known that combination forecasting strategies produce superior out-of-sample forecasting performances. In the M4 forecasting competition, a very simple forecast combination strategy achieved third place on yearly time series. An analysis of the ensemble model and its component models suggests that the competitive accuracy comes from avoiding poor forecasts, rather than from beating the best individual models. Moreover, the simple ensemble model can be fitted very quickly, can easily scale horizontally with additional CPU cores or a cluster of computers, and can be implemented by users very quickly and easily. This approach might be of particular interest to users who need accurate yearly forecasts without being able to spend significant time, resources, or expertise on tuning models. Users of the R statistical programming language can access this modeling approach using the “forecastHybrid” package.     

Diffusion of multi-generational high-technology products

Previous multi-generational product diffusion (MGPD) models were developed based on the diffusion patterns at that time, but may not be adopted in today's cases. By incorporating the effect of customers' forward-looking behaviour, this paper offers a parsimonious and original model that captures the dynamics of MGPD in current high-technology markets. We empirically examine the feasibility of using previous MGPD models and our suggested model to explain the market growth of new products from high-technology industries. The results show that the new model exhibits better curve fitting and forecasting performance than the prior MGPD models in the cases of this study. For marketing researchers, our model and its results suggest customers' forward looking behaviour is perhaps one of the key sales affecting factors that are missing in previous MGPD models in explaining nowadays' cases. For marketing practitioners, this study offers a valuable tool for marketing strategies in high-tech industries.     

Forecasting week-to-week television ratings using reduced-form and structural dynamic models

Rather than being sold several months before a program is aired, more than 20% of TV advertising slots are retained for sale weekly near the program’s broadcast time. Distinct from the literature that mainly focuses on the forecasting of program ratings for advanced sales of advertising slots, we explore approaches that can provide more accurate forecasts for near real-time ratings. We propose two dynamic models that mainly employ individual viewing records for past episodes to forecast viewers’ decisions on episodes in the coming week, and therefore the ratings for these episodes. One is a reduced-form dynamic model that measures the influence of past watching experience by the weighted average of the viewers’ choices of past episodes. The other is a structural dynamic model that goes deeper in its use of previous viewing information by modeling the underlying process of this influence based on the Bayesian updating theory. Using data from the Hong Kong TV industry, we test and compare the two models. Results show that the reduced-form model generally performs better when the variance of ratings across episodes is small, while the structural model generates more accurate forecasts in other cases.     

A disaster response model driven by spatial–temporal forecasts

In this research, we propose a disaster response model combining preparedness and responsiveness strategies. The selective response depends on the level of accuracy that our forecasting models can achieve. In order to decide the right geographical space and time window of response, forecasts are prepared and assessed through a spatial–temporal aggregation framework, until we find the optimum level of aggregation. The research considers major earthquake data for the period 1985–2014. Building on the produced forecasts, we develop accordingly a disaster response model. The model is dynamic in nature, as it is updated every time a new event is added in the database. Any forecasting model can be optimized though the proposed spatial–temporal forecasting framework, and as such our results can be easily generalized. This is true for other forecasting methods and in other disaster response contexts.     

Stability in the inefficient use of forecasting systems: A case study in a supply chain company

Computer-based demand forecasting systems have been widely adopted in supply chain companies, but little research has studied how these systems are actually used in the forecasting process. We report the findings of a case study of demand forecasting in a pharmaceutical company over a 15-year period. At the start of the study, managers believed that they were making extensive use of their forecasting system that was marketed based on the accuracy of its advanced statistical methods. Yet most forecasts were obtained using the system’s facility for judgmentally overriding the automatic statistical forecasts. Carrying out the judgmental interventions involved considerable management effort as part of a sales & operations planning (S&OP) process, yet these often only served to reduce forecast accuracy. This study uses observations of the forecasting process, interviews with participants and data on the accuracy of forecasts to investigate why the managers continued to use non-normative forecasting practices for many years despite the potential economic benefits that could be achieved through change. The reasons for the longevity of these practices are examined both from the perspective of the individual forecaster and the organization as a whole.     

Applicability of the M5 to Forecasting at Walmart

The M5 Forecasting Competition, the fifth in the series of forecasting competitions organized by Professor Spyros Makridakis and the Makridakis Open Forecasting Center at the University of Nicosia, was an extremely successful event. This competition focused on both the accuracy and uncertainty of forecasts and leveraged actual historical sales data provided by Walmart. This has led to the M5 being a unique competition that closely parallels the difficulties and challenges associated with industrial applications of forecasting. Like its precursor the M4, many interesting ideas came from the results of the M5 competition which will continue to push forecasting in new directions.In this article we discuss four topics around the practitioners view of the application of the competition and its results to the actual problems we face. First, we examine the data provided and how it relates to common difficulties practitioners must overcome. Secondly, we review the relevance of the accuracy and uncertainty metrics associated with the competition. Third, we discuss the leading solutions and their implications to forecasting at a company like Walmart. We then close with thoughts about a future M6 competition and further enhancements that can be explored.     

A stochastic differential equation approach to the analysis of the 2017 and 2019 UK general election polls

Human dynamics and sociophysics build on statistical models that can shed light on and add to our understanding of social phenomena. We propose a generative model based on a stochastic differential equation that enables us to model the opinion polls leading up to the 2017 and 2019 UK general elections and to make predictions relating to the actual results of the elections. After a brief analysis of the time series of the poll results, we provide empirical evidence that the gamma distribution, which is often used in financial modelling, fits the marginal distribution of this time series. We demonstrate that the proposed poll-based forecasting model may improve upon predictions based solely on polls. The method uses the Euler–Maruyama method to simulate the time series, measuring the prediction error with the mean absolute error and the root mean square error, and as such could be used as part of a toolkit for forecasting elections.     

Forecasting cryptocurrencies under model and parameter instability

This paper studies the predictability of cryptocurrency time series. We compare several alternative univariate and multivariate models for point and density forecasting of four of the most capitalized series: Bitcoin, Litecoin, Ripple and Ethereum. We apply a set of crypto-predictors and rely on dynamic model averaging to combine a large set of univariate dynamic linear models and several multivariate vector autoregressive models with different forms of time variation. We find statistically significant improvements in point forecasting when using combinations of univariate models, and in density forecasting when relying on the selection of multivariate models. Both schemes deliver sizable directional predictability.     

Advertising and Coordination in Markets with Consumption Scale Effects

Many advertised products are established and have little quality variation. For these products advertising signaling explanations are unconvincing. We develop a coordination model of advertising with consumers observing ads probabilistically and never observing advertising levels. Consumers who fail to see an ad for a product believe it will likely have low sales and so be of low value. Firms advertise to avoid these beliefs. The model's predictions on advertising, market share, and profitability are consistent with observed outcomes. The model produces the time series behavior for prices and market share observed in the data and not available from existing coordination models.     

Forecasting crude oil prices with DSGE models

In this study, we conducted an oil prices forecasting competition among a set of structural models, including vector autoregression and dynamic stochastic general equilibrium (DSGE) models. Our results highlight two principles. First, forecasts should exploit the fact that real oil prices are mean reverting over long horizons. Second, models should not replicate the high volatility of the oil prices observed in samples. By following these principles, we show that an oil sector DSGE model performs much better at real oil price forecasting than random walk or vector autoregression.     

Penalized estimation of panel vector autoregressive models: A panel LASSO approach

This paper proposes LASSO estimation specific for panel vector autoregressive (PVAR) models. The penalty term allows for shrinkage for different lags, for shrinkage towards homogeneous coefficients across panel units, for penalization of lags of variables belonging to another cross-sectional unit, and for varying penalization across equations. The penalty parameters therefore build on time series and cross-sectional properties that are commonly found in PVAR models. Simulation results point towards advantages of using the proposed LASSO for PVAR models over ordinary least squares in terms of forecast accuracy. An empirical forecasting application including 20 countries supports these findings.