A brief history of forecasting competitions

Forecasting competitions are now so widespread that it is often forgotten how controversial they were when first held, and how influential they have been over the years. I briefly review the history of forecasting competitions, and discuss what we have learned about their design and implementation, and what they can tell us about forecasting. I also provide a few suggestions for potential future competitions, and for research about forecasting based on competitions.     

The Marshallian macroeconomic model: A progress report

In this progress report, we first indicate the origins and early development of the Marshallian macroeconomic model (MMM) and briefly review some of our past empirical forecasting experiments with the model. Then we present recently developed one-sector, two-sector and n-sector models of an economy that can be employed to explain past experience, predict future outcomes and analyze policy problems. The results of simulation experiments with various versions of the model are provided to illustrate some of its dynamic properties that include “chaotic” features. Last, we present comments on planned future work with the model.     

Forecasting for COVID-19 has failed

Epidemic forecasting has a dubious track-record, and its failures became more prominent with COVID-19. Poor data input, wrong modeling assumptions, high sensitivity of estimates, lack of incorporation of epidemiological features, poor past evidence on effects of available interventions, lack of transparency, errors, lack of determinacy, consideration of only one or a few dimensions of the problem at hand, lack of expertise in crucial disciplines, groupthink and bandwagon effects, and selective reporting are some of the causes of these failures. Nevertheless, epidemic forecasting is unlikely to be abandoned. Some (but not all) of these problems can be fixed. Careful modeling of predictive distributions rather than focusing on point estimates, considering multiple dimensions of impact, and continuously reappraising models based on their validated performance may help. If extreme values are considered, extremes should be considered for the consequences of multiple dimensions of impact so as to continuously calibrate predictive insights and decision-making. When major decisions (e.g. draconian lockdowns) are based on forecasts, the harms (in terms of health, economy, and society at large) and the asymmetry of risks need to be approached in a holistic fashion, considering the totality of the evidence.     

Forecasting in humanitarian operations: Literature review and research needs

Forecasting research in the humanitarian context is scarce. In this literature review, our goal is not only to show why forecasting research is important for the humanitarian sector, but also to identify what has been done so far, and where are the needs for further research. We conducted a structured literature search in Scopus, Web of Science, ABI Inform, and Google Scholar resulted in only 38 papers published between 1990 and 2018. Based on our findings we highlight three case studies as exemplary research in forecasting within the humanitarian context and list seven future research streams with specific research needs identified in each stream.     

Developing breakthrough products: challenges and options for market assessment

Competitive pressures and market forces are augmenting the importance of product innovation as a source of competitive advantage. Key drivers underpinning market success have the capacity to develop the “right” products for the “right” customers, using the “right” channels, with a shorter development cycle than competitors. But when breakthrough products (BTPs) are involved, how do we sort out what is “right”? How do we identify customers' future requirements when the technology platforms are still unfolding, products are still in development and customers lack experience with the product? Traditional product development market research techniques are problematic since they assume the customer has historical experience with products similar to those being created. The objective of this paper is to raise the awareness of the OM community with the strengths and weaknesses of existing methodologies for developing and introducing innovative products to market. Further, it suggests promising future directions for both practice and research when BTPs are involved. This is important because of the central role OM plays in managing the success of the BTP development process.     

Forecasting in social settings: The state of the art

This paper provides a non-systematic review of the progress of forecasting in social settings. It is aimed at someone outside the field of forecasting who wants to understand and appreciate the results of the M4 Competition, and forms a survey paper regarding the state of the art of this discipline. It discusses the recorded improvements in forecast accuracy over time, the need to capture forecast uncertainty, and things that can go wrong with predictions. Subsequently, the review classifies the knowledge achieved over recent years into (i) what we know, (ii) what we are not sure about, and (iii) what we don’t knowIn the first two areas, we explore the difference between explanation and prediction, the existence of an optimal model, the performance of machine learning methods on time series forecasting tasks, the difficulties of predicting non-stable environments, the performance of judgment, and the value added by exogenous variables. The article concludes with the importance of (thin and) fat tails, the challenges and advances in causal inference, and the role of luck.     

How similar are they? rethinking electoral congruence

Electoral continuity and discontinuity have been a staple of voting research for decades. Most researchers have employed Pearson’s r as a measure of congruence between two electoral outcomes across a set of geographic units. This paper argues that that practice should be abandoned. The correlation coefficient is a measure of linearity, not similarity, and is almost always the wrong measure. The paper recommends other quantities that better accord with what researchers usually mean by electoral persistence. Replications of prior studies in American and comparative politics demonstrate that the consequences of using r when it is inappropriate can be stark. In some cases what we think are continuities are actually discontinuities.     

Approximate Bayesian forecasting

Approximate Bayesian Computation (ABC) has become increasingly prominent as a method for conducting parameter inference in a range of challenging statistical problems, most notably those characterized by an intractable likelihood function. In this paper, we focus on the use of ABC not as a tool for parametric inference, but as a means of generating probabilistic forecasts; or for conducting what we refer to as ‘approximate Bayesian forecasting’. The four key issues explored are: (i) the link between the theoretical behavior of the ABC posterior and that of the ABC-based predictive; (ii) the use of proper scoring rules to measure the (potential) loss of forecast accuracy when using an approximate rather than an exact predictive; (iii) the performance of approximate Bayesian forecasting in state space models; and (iv) the use of forecasting criteria to inform the selection of ABC summaries in empirical settings. The primary finding of the paper is that ABC can provide a computationally efficient means of generating probabilistic forecasts that are nearly identical to those produced by the exact predictive, and in a fraction of the time required to produce predictions via an exact method.     

Forecasting spare part demand with installed base information: A review

The classical spare part demand forecasting literature studies methods for forecasting intermittent demand. However, the majority of these methods do not consider the underlying demand-generating factors. The demand for spare parts originates from the replacement of parts in the installed base of machines, either preventively or upon breakdown of the part. This information from service operations, which we refer to as installed base information, can be used to forecast the future demand for spare parts. This paper reviews the literature on the use of such installed base information for spare part demand forecasting in order to asses (1) what type of installed base information can be useful; (2) how this information can be used to derive forecasts; (3) the value of using installed base information to improve forecasting; and (4) the limits of the existing methods. This serves as motivation for future research.     

Health and work of the elderly: subjective health measures,reporting errors and endogeneity in the relationship between health and work

This paper explores the interrelation between health and work decisions of older workers. For this, two issues are of relevance. Firstly, health and work may be endogenously related because of direct causal effects of health on work and vice versa, and because of unobservables that may affect both observed health and work outcomes. Secondly, social surveys usually contain self‐assessed health measures and research indicates that these may be subject to endogenous, state‐dependent reporting bias. A solution to the ‘Health and Retirement Nexus’ therefore requires an integrated model for work decisions, health production and health reporting mechanisms. We formulate such a model and estimate it on a longitudinal dataset of older Dutch males. Copyright © 2009 John Wiley & Sons, Ltd.     

The economic and financial properties of crude oil: A review

In this article, we provide a structured review of crude oil price dynamics. Specifically, we summarize evidence on important factors determining oil prices, cover the impact of oil market shocks on the macro economy and the stock market, discuss how the financialization of crude oil markets affects oil market functionality and efficiency, and we then outline approaches for forecasting crude oil prices and volatility. By comparing the results of the most influential early contributions and recent studies, we can identify important developments and research gaps in each field. Thus, our review provides academics and practitioners newly engaging in crude oil research with an overview of what scientists know about crude oil dynamics and highlights which topics areparticularly promising for future research.     

Density forecasting with Bayesian Vector Autoregressive models under macroeconomic data uncertainty

Macroeconomic data are subject to data revisions. Yet, the usual way of generating real-time density forecasts from Bayesian Vector Autoregressive (BVAR) models makes no allowance for data uncertainty from future data revisions. We develop methods of allowing for data uncertainty when forecasting with BVAR models with stochastic volatility. First, the BVAR forecasting model is estimated on real-time vintages. Second, the BVAR model is jointly estimated with a model of data revisions such that forecasts are conditioned on estimates of the ‘true’ values. We find that this second method generally improves upon conventional practice for density forecasting, especially for the United States.     

Towards an understanding of the female expatriate experience in Europe

The increased participation of women in the workforce has been one of the major changes in the structuring of the labour force in recent years, and it is anticipated that this trend will continue. Despite growing numbers of women in senior domestic management roles, the participation rates of women in international management remains low across the globe. In Europe, international management has generally been a masculine preserve. Much of our knowledge of female expatriation comes from North America and is based on the experiences of female managers working for North American MNCs. This article builds on that base of understanding but highlights a growing stream of research into female expatriation in Europe, which remains largely “invisible” to specialists outside Europe. Given the paucity of empirical research in this area in general and the need for a more international understanding of the phenomena which can arise from examining different contexts, the article suggests that researchers outside Europe may find useful insights in this paper which pulls together and summarises what we know from the existing research on female expatriates in Europe and offers some avenues for future research.     

Measuring health expenditures and outcomes in saarc region: health is a luxury?

The objective of the study is to investigate the link between health care expenditures (HCE), economic growth and health outcomes (i.e., life expectancy (LE), infant mortality and the share of elderly people) for a panel of South Asian Association for Regional Cooperation (SAARC) countries; namely, Afghanistan, Bangladesh, Bhutan, India, Maldives, Nepal, Pakistan and Sri Lanka; over the period of 1995–2010. The developed panel cointegration technique is employed for analysis of short and long-run relationship between the variables. The results of panel cointegration found that there is a long-run relationship between health expenditures, economic growth and health outcomes in SAARC region. The estimated results indicate that both, LE and share of elderly people in population have a negative relationship related to the HCEs which signifies that HCEs are luxury goods in SAARC countries. The burden of aging population can be lowered by provision of quality health services and utilization of their experience and knowledge in dynamics of economic development forecasting. Furthermore, there is no significant relationship found between infant mortality rate (IMR) and HCEs which implies that HCEs do not provide sufficient benefits to reduce infant mortality. Similarly, HCEs are not enough in generating GDP, lowering IMR and increasing LE in SAARC region. The implementation of appropriate tax reform, stable food prices and trade promotion for low transaction cost medical equipment is required for the SAARC region.     

Assessment of meaningful change in routine outcome measurement (ROM) with a combination of a longitudinal and a ‘classify and count’ approach

To assess significant changes of health status in people receiving health care, distribution-based and anchor-based methods have been proposed. However, there is no real consensus on what method is the best for evaluating clinically meaningful change. To maximize the internal and external validity of outcome assessment, we propose combining two approaches as recommended by recent practical guidelines on this field. Specifically, we suggest applying longitudinal hierarchical linear models on subgroups of patients showing reliable change and reliable and clinically significant change. This combined approach improved the model’s ability (1) to quantify the magnitude of changes to be reliable and clinically meaningful and (2) to select significant predictors of changes. An empirical application on a prevalence sample of Italian outpatients attending four community mental health services was done. A cross-sectional model and three longitudinal models were applied on the entire study sample and reliable and clinically meaningful change subsamples to investigate the magnitude of change and the predictive effect on outcomes of clinical, socio-demographic and process variables on different patients’ subgroups. Differences were found suggesting that both the statistical method and the sample used to calculate individual changes affect the estimates. The main conclusion is that ignoring the longitudinal data structure or including patients with unreliable change at the follow-up might result in misleading inferences that can alter the real magnitude of changes and the contributions of predictors. The approach proposed provides robust feedback to clinicians on clinically significant change and can be recommended in outcome studies and research.     

Conducting a sales forecasting audit

Continuous improvement in sales forecasting is a worthy goal for any organization. This paper describes a methodology for conducting a sales forecasting audit, the goal of which is to help a company understand the status of its sales forecasting processes and identify ways to improve those processes. The methodology described here has been developed over a 5-year period, involving multiple auditors, and has been implemented (to date) at 16 organizations. This methodology revolves around three distinct phases: the ‘as-is’ phase, in which the audit team seeks to understand fully a company’s current forecasting process; the ‘should-be’ phase, in which the audit team presents a vision of what world-class forecasting should look like at the audited company, and the ‘way-forward’ phase, in which the audit team presents a roadmap of how the company can change its forecasting processes to achieve world-class levels. Those companies that have responded positively to the audit process have experienced significant improvement in their forecasting performance. The paper concludes by presenting lessons from audits conducted to date, as well as implications for management practice and future research.     

Affection not affliction: The role of emotions in information systems and organizational change

Most IS research in both the technical/rational and socio-technical traditions ignores or marginalizes the emotionally charged behaviours through which individuals engage in, and cope with the consequences of, IS practice and associated organizational change. Even within the small body of work that engages with emotions through particular conceptual efforts, affections are often conceived as a phenomenon to be eradicated – an affliction requiring a cure. In this paper, I argue that emotions are always implicated in our lived experiences, crucially influencing how we come to our beliefs about what is good or bad, right or wrong. I draw from the theoretical work of Michel Foucault to argue for elaborating current notions of IS innovation as a moral and political struggle in which individuals’ beliefs and feelings are constantly tested. Finally, I demonstrate these ideas by reference to a case study that had considerable emotional impact, and highlight the implications for future work.     

Machine learning in M4: What makes a good unstructured model?

The Makridakis Competitions seek to identify the most accurate forecasting methods for different types of predictions. The M4 competition was the first in which a model of the type commonly described as “machine learning” has outperformed the more traditional statistical approaches, winning the competition. However, many approaches that were self-labeled as “machine learning” failed to produce accurate results, which generated discussion about the respective benefits and drawbacks of “statistical” and “machine learning” approaches. Both terms have remained ill-defined in the context of forecasting. This paper introduces the terms “structured” and “unstructured” models to better define what is intended by the use of the terms “statistical” and “machine learning” in the context of forecasting based on the model’s data generating process. The mechanisms that underlie specific challenges to unstructured modeling are examined in the context of forecasting, along with common solutions. Finally, the innovations in the winning model that allowed it to overcome these challenges and produce highly accurate results are highlighted.