Individual versus group spot price forecasting in the international petroleum market: A case study

This paper utilizes the conventional statistical tests associated with the rational expectations hypothesis so as to compare the relative accuracy of individual versus group forecasting within the organization. In order to maintain comparability between forecasting regimens the study employs like information sets for the two prediction methods. Using the rational expectations tests as criteria, the statistical results show group forecasts inferior to individually produced predictions These findings imply that group-produced forecasting accuracy may be hampered by the psychological interaction associated with consensus behavior. Conversely, we find forecasting accuracy improves when predictions are elicited from individuals in an isolated laboratory-like setting.     

Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task

We conducted laboratory experiments for analyzing the accuracy of three structured approaches (nominal groups, Delphi, and prediction markets) relative to traditional face-to-face meetings (FTF). We recruited 227 participants (11 groups per method) who were required to solve a quantitative judgment task that did not involve distributed knowledge. This task consisted of ten factual questions, which required percentage estimates. While we did not find statistically significant differences in accuracy between the four methods overall, the results differed somewhat at the individual question level. Delphi was as accurate as FTF for eight questions and outperformed FTF for two questions. By comparison, prediction markets did not outperform FTF for any of the questions and were inferior for three questions. The relative performances of nominal groups and FTF were mixed and the differences were small. We also compared the results from the three structured approaches to prior individual estimates and staticized groups. The three structured approaches were more accurate than participants’ prior individual estimates. Delphi was also more accurate than staticized groups. Nominal groups and prediction markets provided little additional value relative to a simple average of the forecasts. In addition, we examined participants’ perceptions of the group and the group process. The participants rated personal communications more favorably than computer-mediated interactions. The group interactions in FTF and nominal groups were perceived as being highly cooperative and effective. Prediction markets were rated least favourably: prediction market participants were least satisfied with the group process and perceived their method as the most difficult.     

Role thinking: Standing in other people’s shoes to forecast decisions in conflicts

When forecasting decisions in conflict situations, experts are often advised to figuratively stand in the other person’s shoes. We refer to this as “role thinking”, because, in practice, the advice is to think about how other protagonists will view the situation in order to predict their decisions. We tested the effect of role thinking on forecast accuracy. We obtained 101 role-thinking forecasts of the decisions that would be made in nine diverse conflicts from 27 Naval postgraduate students (experts) and 107 role-thinking forecasts from 103 second-year organizational behavior students (novices). The accuracy of the novices’ forecasts was 33% and that of the experts’ was 31%; both were little different from chance (guessing), which was 28%. The small improvement in accuracy from role-thinking strengthens the finding from earlier research that it is not sufficient to think hard about a situation in order to predict the decisions which groups of people will make when they are in conflict. Instead, it is useful to ask groups of role players to simulate the situation. When groups of novice participants adopted the roles of protagonists in the aforementioned nine conflicts and interacted with each other, their group decisions predicted the actual decisions with an accuracy of 60%.     

Forecasting from time series subject to sporadic perturbations: Effectiveness of different types of forecasting support

How effective are different approaches for the provision of forecasting support? Forecasts may be either unaided or made with the help of statistical forecasts. In practice, the latter are often crude forecasts that do not take sporadic perturbations into account. Most research considers forecasts based on series that have been cleansed of perturbation effects. This paper considers an experiment in which people made forecasts from time series that were disturbed by promotions. In all conditions, under-forecasting occurred during promotional periods and over-forecasting during normal ones. The relative sizes of these effects depended on the proportions of periods in the data series that contained promotions. The statistical forecasts improved the forecasting accuracy, not because they reduced these biases, but because they decreased the random error (scatter). The performance improvement did not depend on whether the forecasts were based on cleansed series. Thus, the effort invested in producing cleansed time series from which to forecast may not be warranted: companies may benefit from giving their forecasters even crude statistical forecasts. In a second experiment, forecasters received optimal statistical forecasts that took the effects of promotions into account fully. This increased the accuracy because the biases were almost eliminated and the random error was reduced by 20%. Thus, the additional effort required to produce forecasts that take promotional effects into account is worthwhile.     

Forecasts of US short-term interest rates: A flexible forecast combination approach

This paper develops a flexible approach to combine forecasts of future spot rates with forecasts from time-series models or macroeconomic variables. We find empirical evidence that, accounting for both regimes in interest rate dynamics, and combining forecasts from different models, helps improve the out-of-sample forecasting performance for US short-term rates. Imposing restrictions from the expectations hypothesis on the forecasting model are found to help at long forecasting horizons.     

Combination of long term and short term forecasts, with application to tourism demand forecasting

Forecast combination is a well-established and well-tested approach for improving the forecasting accuracy. One beneficial strategy is to use constituent forecasts that have diverse information. In this paper we consider the idea of diversity being accomplished by using different time aggregations. For example, we could create a yearly time series from a monthly time series and produce forecasts for both, then combine the forecasts. These forecasts would each be tracking the dynamics of different time scales, and would therefore add diverse types of information. A comparison of several forecast combination methods, performed in the context of this setup, shows that this is indeed a beneficial strategy and generally provides a forecasting performance that is better than the performances of the individual forecasts that are combined.As a case study, we consider the problem of forecasting monthly tourism numbers for inbound tourism to Egypt. Specifically, we consider 33 individual source countries, as well as the aggregate. The novel combination strategy also produces a generally improved forecasting accuracy.     

Stability in the inefficient use of forecasting systems: A case study in a supply chain company

Computer-based demand forecasting systems have been widely adopted in supply chain companies, but little research has studied how these systems are actually used in the forecasting process. We report the findings of a case study of demand forecasting in a pharmaceutical company over a 15-year period. At the start of the study, managers believed that they were making extensive use of their forecasting system that was marketed based on the accuracy of its advanced statistical methods. Yet most forecasts were obtained using the system’s facility for judgmentally overriding the automatic statistical forecasts. Carrying out the judgmental interventions involved considerable management effort as part of a sales & operations planning (S&OP) process, yet these often only served to reduce forecast accuracy. This study uses observations of the forecasting process, interviews with participants and data on the accuracy of forecasts to investigate why the managers continued to use non-normative forecasting practices for many years despite the potential economic benefits that could be achieved through change. The reasons for the longevity of these practices are examined both from the perspective of the individual forecaster and the organization as a whole.     

Long-range U.S. Manpower forecasts in retrospect: How accurate were we?

This paper compares and evaluates the accuracy of long-range occupational manpower forecasts made for 1980 in the early 1970s by the U.S. Bureau of Labor Statistics and by the author. The different assumptions and forecasting methodologies utilized are discussed, and the occupational forecasts are then compared to the actual 1980 employment data. The relative accuracy of the different sets of forecasts is assessed according to several different criteria, and the larger question of the usefulness of either set of forecasts is addressed. It is found that neither set of forecasts was clearly superior, that the accuracy of both sets of forecasts was generally poor, and that the projections for individual occupations were often so wide of the mark as to be of questionable usefulness for manpower planning and vocational guidance. The implications of these findings for manpower forecasting are discussed.The author is grateful to several referees for helpful comments on an earlier draft of this paper, but retains sole responsibility for the opinions expressed here and for any errors.     

Co-integration constraint and forecasting: An empirical examination

Does co-integration help long-term forecasts? In this paper, we use simulation, real data sets, and multi-step-ahead post-sample forecasts to study this question. Based on the square root of the trace of forecasting error-covariance matrix, we found that for simulated data imposing the ‘correct’ unit-root constraints implied by co-integration does improve the accuracy of forecasts. For real data sets, the answer is mixed. Imposing unit-root constraints suggested by co-integration tests produces better forecasts for some cases, but fares poorly for others. We give some explanations for the poor performance of co-integration in long-term forecasting and discuss the practical implications of the study. Finally, an adaptive forecasting procedure is found to perform well in one- to ten-step-ahead forecasts.     

Applications of machine learning for corporate bond yield spread forecasting

This article considers nine different predictive techniques, including state-of-the-art machine learning methods for forecasting corporate bond yield spreads with other input variables. We examine each method’s out-of-sample forecasting performance using two different forecast horizons: (1) the in-sample dataset over 2003–2007 is used for one-year-ahead and two-year-ahead forecasts of non-callable corporate bond yield spreads; and (2) the in-sample dataset over 2003–2008 is considered to forecast the yield spreads in 2009. Evaluations of forecasting accuracy have shown that neural network forecasts are superior to the other methods considered here in both the short and longer horizon. Furthermore, we visualize the determinants of yield spreads and find that a firm’s equity volatility is a critical factor in yield spreads.     

Hierarchical forecasting with a top-down alignment of independent-level forecasts

Hierarchical forecasting with intermittent time series is a challenge in both research and empirical studies. Extensive research focuses on improving the accuracy of each hierarchy, especially the intermittent time series at bottom levels. Then, hierarchical reconciliation can be used to improve the overall performance further. In this paper, we present a hierarchical-forecasting-with-alignment approach that treats the bottom-level forecasts as mutable to ensure higher forecasting accuracy on the upper levels of the hierarchy. We employ a pure deep learning forecasting approach, N-BEATS, for continuous time series at the top levels, and a widely used tree-based algorithm, LightGBM, for intermittent time series at the bottom level. The hierarchical-forecasting-with-alignment approach is a simple yet effective variant of the bottom-up method, accounting for biases that are difficult to observe at the bottom level. It allows suboptimal forecasts at the lower level to retain a higher overall performance. The approach in this empirical study was developed by the first author during the M5 Accuracy competition, ranking second place. The method is also business orientated and can be used to facilitate strategic business planning.     

The performance of the global bottom-up approach in the M5 accuracy competition: A robustness check

The M5 accuracy competition has presented a large-scale hierarchical forecasting problem in a realistic grocery retail setting in order to evaluate an extended range of forecasting methods, particularly those adopting machine learning. The top ranking solutions adopted a global bottom-up approach, by which is meant using global forecasting methods to generate bottom level forecasts in the hierarchy and then using a bottom-up strategy to obtain coherent forecasts for aggregate levels. However, whether the observed superior performance of the global bottom-up approach is robust over various test periods or only an accidental result, is an important question for retail forecasting researchers and practitioners. We conduct experiments to explore the robustness of the global bottom-up approach, and make comments on the efforts made by the top-ranking teams to improve the core approach. We find that the top-ranking global bottom-up approaches lack robustness across time periods in the M5 data. This inconsistent performance makes the M5 final rankings somewhat of a lottery. In future forecasting competitions, we suggest the use of multiple rolling test sets to evaluate the forecasting performance in order to reward robustly performing forecasting methods, a much needed characteristic in any application.     

Forecasting sales response for multiple time horizons and temporally aggregated data: A comparison of constant and stochastic coefficient models

Past research on time-varying sales-response models emphasized the application of different estimation techniques in examining variation in advertising effectiveness over time. This study focuses on comparing sales forecasts using constant and stochastic coefficients sales-response models. Selected constant and stochastic coefficient models are applied to six sets of bimonthly and one set of annual advertising and sales data to assess forecasting accuracy for time horizons of various lengths. Results show improved forecasting accuracy for a first-order autoregressive stochastic coefficient model, particularly in short-run forecasting applications.     

Taking a Second Look in a Warped Crystal Ball: Explaining the Accuracy of Revised Forecasts

The fundamental questions we address are whether firms with a higher initial forecasting ability are able to accurately revise the exit forecasts of their investments; and how co‐investment partners and value‐adding commitment with their investment influence the main effect. We explore these questions with novel and unique data collected via mixed research methods on venture capital firms’ forecasts of 114 portfolio companies. We find that venture capital firms that are better at making initial forecasts are less effective in revising their forecasts. In addition, while the number of co‐investment partners positively moderate this relationship, venture capital firms’ value‐adding commitment moderates it negatively. Our findings contribute to the literature on organizational forecasting as well as inter‐organizational knowledge transfer and knowledge creation. They also provide novel insights into venture capital literature and practice.     

Evaluation of the best M4 competition methods for small area population forecasting

The ‘M4’ forecasting competition results were featured recently in a special issue of the International Journal of Forecasting and included projections for demographic time series. We sought to investigate whether the best M4 methods could improve the accuracy of small area population forecasts, which generally suffer from much higher forecast errors than regions with larger populations. The aim of this study was to apply the top ten M4 forecasting methods to produce 5- and 10-year forecasts of small area total populations using historical datasets from Australia and New Zealand. Forecasts were compared against the actual population numbers and forecasts from two simple benchmark models. The M4 methods were found to perform relatively well compared to our benchmarks. In the light of these findings, we discuss possible future directions for small area population forecasting research.     

Business and consumer expectations and macroeconomic forecasts

Business and consumer surveys have become an essential tool for gathering information about different economic variables. While the fast availability of the results and the wide range of variables covered have made them very useful for monitoring the current state of the economy, there is no consensus on their usefulness for forecasting macroeconomic developments.The objective of this paper is to analyse the possibility of improving forecasts for selected macroeconomic variables for the euro area using the information provided by these surveys. After analyzing the potential presence of seasonality and the issue of quantification, we tested whether these indicators provide useful information for improving forecasts of the macroeconomic variables. With this aim, different sets of models have been considered (AR, ARIMA, SETAR, Markov switching regime models and VAR) to obtain forecasts for the selected macroeconomic variables. Then, information from surveys has been considered for forecasting these variables in the context of the following models: autoregressive, VAR, Markov switching regime and leading indicator models. In all cases, the root mean square error (RMSE) has been computed for different forecast horizons.The comparison of the forecasting performance of the two sets of models permits us to conclude that, in most cases, models that include information from the surveys have lower RMSEs than the best model without survey information. However, this reduction is only significant in a limited number of cases. In this sense, the results obtained extend the results of previous research that has included information from business and consumer surveys to explain the behaviour of macroeconomic variables, but are not conclusive about its role.     

Group-based forecasting?: A social psychological analysis

The potential of group (vs. individual) forecasting is analyzed from the perspective of the social psychology of groups. The social decision scheme theory (SDST) is summarized, and several simulations are presented to demonstrate the dependence of group aggregation accuracy upon factors such as group size, the accuracy and distribution of individual forecasts, and shared representations of the forecasting problem. Many advantages and disadvantages of group aggregation are identified and related to four generic methods of group aggregation (statistical aggregation, prediction markets, the Delphi method, and face-to-face discussion). A number of aspects of forecasting problems are identified which should govern whether or not group forecasting can be relied upon, and if so, what aggregation method should be used.     

A disaster response model driven by spatial–temporal forecasts

In this research, we propose a disaster response model combining preparedness and responsiveness strategies. The selective response depends on the level of accuracy that our forecasting models can achieve. In order to decide the right geographical space and time window of response, forecasts are prepared and assessed through a spatial–temporal aggregation framework, until we find the optimum level of aggregation. The research considers major earthquake data for the period 1985–2014. Building on the produced forecasts, we develop accordingly a disaster response model. The model is dynamic in nature, as it is updated every time a new event is added in the database. Any forecasting model can be optimized though the proposed spatial–temporal forecasting framework, and as such our results can be easily generalized. This is true for other forecasting methods and in other disaster response contexts.     

A comparison of the forecasting performance of WEFA and ARIMA time series methods

This paper examines the forecasting performance of the Wharton model (MARK III) over the period 1973 through 1975 and compares it with that of ARIMA models' performance over the same period. Despite strong intimation in the literature to the contrary, we find that this econometric model, at least, exhibits greater accuracy in every respect relative to ARIMA methods, in terms of its forecasts cum constant adjustments. When constant adjustments are disallowed then its forecasts are still more accurate than ARIMA forecasts over a 4- and 8-quarter forecasting horizon, but less accurate over a 1-quarter horizon. The comparison was carried out over twenty three macrovariables, under a slight handicap for the Wharton Model, in that the latter's parameters were estimated over a sample ending in 1969.3 while the ARIMA models were reidentified and reestimated as of the quarter immediately preceding the forecast.     

Evaluating quantile forecasts in the M5 uncertainty competition

Probabilistic forecasts are necessary for robust decisions in the face of uncertainty. The M5 Uncertainty competition required participating teams to forecast nine quantiles for unit sales of various products at various aggregation levels and for different time horizons. This paper evaluates the forecasting performance of the quantile forecasts at different aggregation levels and at different quantile levels. We contrast this with some theoretical predictions, and discuss potential implications and promising future research directions for the practice of probabilistic forecasting.