The impossibility of accurate macro-economic forecasting

The macro-economic short-term forecasting record in the West over the past thirty years is very poor. Modern non-linear signal processing techniques can be used to show that such inaccuracy is a deep and inherent property of the data themselves. The forecasting record simply cannot be improved. Much economic policy still focuses on short-term intervention based on short-term forecasts. But such efforts are futile because forecasts of sufficient accuracy over time cannot be made.     

Empirical tests of input-output forecasts

This paper analyzes previous studies of the accuracy of input-output forecasts as compared with projections derived from alternative forecasting techniques. The problem of constructing appropriate tests of input-output forecasts is discussed. Major tests of the interindustry approach and alternative techniques, such as final demand blowup, GNP blowup and multiple regression, conducted in the past four decades are reviewed and the major findings summarized. It is shown here that, contrary to the belief of some economists, the input-output forecasting model performs as well as and usually better than any of the alternatives considered.     

Online hierarchical forecasting for power consumption data

This paper proposes a three-step approach to forecasting time series of electricity consumption at different levels of household aggregation. These series are linked by hierarchical constraints—global consumption is the sum of regional consumption, for example. First, benchmark forecasts are generated for all series using generalized additive models. Second, for each series, the aggregation algorithm ML-Poly, introduced by Gaillard, Stoltz, and van Erven in 2014, finds an optimal linear combination of the benchmarks. Finally, the forecasts are projected onto a coherent subspace to ensure that the final forecasts satisfy the hierarchical constraints. By minimizing a regret criterion, we show that the aggregation and projection steps improve the root mean square error of the forecasts. Our approach is tested on household electricity consumption data; experimental results suggest that successive aggregation and projection steps improve the benchmark forecasts at different levels of household aggregation.     

Forecasting football match results in national league competitions using score-driven time series models

We develop a new dynamic multivariate model for the analysis and forecasting of football match results in national league competitions. The proposed dynamic model is based on the score of the predictive observation mass function for a high-dimensional panel of weekly match results. Our main interest is in forecasting whether the match result is a win, a loss or a draw for each team. The dynamic model for delivering such forecasts can be based on three different dependent variables: the pairwise count of the number of goals, the difference between the numbers of goals, or the category of the match result (win, loss, draw). The different dependent variables require different distributional assumptions. Furthermore, different dynamic model specifications can be considered for generating the forecasts. We investigate empirically which dependent variable and which dynamic model specification yield the best forecasting results. We validate the precision of the resulting forecasts and the success of the forecasts in a betting simulation in an extensive forecasting study for match results from six large European football competitions. Finally, we conclude that the dynamic model for pairwise counts delivers the most precise forecasts while the dynamic model for the difference between counts is most successful for betting, but that both outperform benchmark and other competing models.     

Mind the gap: A multi-country BVAR benchmark for the Eurosystem projections

The Eurosystem staff forecasts are conditional on the financial markets, the global economy and fiscal policy outlook, and include expert judgement. We develop a multi-country BVAR for the four largest countries of the euro area and we show that it provides accurate conditional forecasts of policy relevant variables such as, for example, consumer prices and GDP. The forecasting accuracy and the ability to mimic the path of the Eurosystem projections suggest that the model is a valid benchmark to assess the consistency of the projections with the conditional assumptions. As such, the BVAR can be used to identify possible sources of judgement, based on the gaps between the Eurosystem projections and the historical regularities captured by the model.     

Long-range U.S. Manpower forecasts in retrospect: How accurate were we?

This paper compares and evaluates the accuracy of long-range occupational manpower forecasts made for 1980 in the early 1970s by the U.S. Bureau of Labor Statistics and by the author. The different assumptions and forecasting methodologies utilized are discussed, and the occupational forecasts are then compared to the actual 1980 employment data. The relative accuracy of the different sets of forecasts is assessed according to several different criteria, and the larger question of the usefulness of either set of forecasts is addressed. It is found that neither set of forecasts was clearly superior, that the accuracy of both sets of forecasts was generally poor, and that the projections for individual occupations were often so wide of the mark as to be of questionable usefulness for manpower planning and vocational guidance. The implications of these findings for manpower forecasting are discussed.The author is grateful to several referees for helpful comments on an earlier draft of this paper, but retains sole responsibility for the opinions expressed here and for any errors.     

A hybrid system-identification method for forecasting telecommunications product demands

A crucial challenge for telecommunications companies is how to forecast changes in demand for specific products over the next 6 to 18 months—the length of a typical short-range capacity-planning and capital-budgeting planning horizon. The problem is especially acute when only short histories of product sales are available. This paper presents a new two-level approach to forecasting demand from short-term data. The lower of the two levels consists of adaptive system-identification algorithms borrowed from signal processing, especially, Hidden Markov Model (HMM) methods [Hidden Markov Models: Estimation and Control (1995) Springer Verlag]. Although they have primarily been used in engineering applications such as automated speech recognition and seismic data processing, HMM techniques also appear to be very promising for predicting probabilities of individual customer behaviors from relatively short samples of recent product-purchasing histories. The upper level of our approach applies a classification tree algorithm to combine information from the lower-level forecasting algorithms. In contrast to other forecast-combination algorithms, such as weighted averaging or Bayesian aggregation formulas, the classification tree approach exploits high-order interactions among error patterns from different predictive systems. It creates a hybrid, forecasting algorithm that out-performs any of the individual algorithms on which it is based. This tree-based approach to hybridizing forecasts provides a new, general way to combine and improve individual forecasts, whether or not they are based on HMM algorithms. The paper concludes with the results of validation tests. These show the power of HMM methods to forecast what individual customers are likely to do next. They also show the gain from classification tree post-processing of the predictions from lower-level forecasts. In essence, these techniques enhance the limited techniques available for new product forecasting.     

Managerial evaluation of exogenous forecast sources

This paper concerns the managerial evaluation of forecast vendors—individuals or firms offering for sale future forecasts of random variables relevant to managerial decision making. Assuming the forecasts are exogenous in the sense they are generated by a methodology unknown or unproven to management, the paper uses a logistic regression model to present a statistical test for informativeness that allows for an interpretation of the vendor's abilities. The advantage of the approach is that it requires as input only knowledge of the unconditional probability distribution of the variable being forecast and a relatively small historical track record of the vendor's forecasting performance. No benchmark forecast is necessary and few assumptions are required about the statistical process that generates the forecasts. As an illustrative empirical application, the paper presents an evaluation of the informativeness of the published long-range price forecasts by a veteran analyst of the Iowa hog market.     

DeepAR: Probabilistic forecasting with autoregressive recurrent networks

Probabilistic forecasting, i.e., estimating a time series’ future probability distribution given its past, is a key enabler for optimizing business processes. In retail businesses, for example, probabilistic demand forecasts are crucial for having the right inventory available at the right time and in the right place. This paper proposes DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an autoregressive recurrent neural network model on a large number of related time series. We demonstrate how the application of deep learning techniques to forecasting can overcome many of the challenges that are faced by widely-used classical approaches to the problem. By means of extensive empirical evaluations on several real-world forecasting datasets, we show that our methodology produces more accurate forecasts than other state-of-the-art methods, while requiring minimal manual work.     

Forecasting macroeconomic time series: LASSO-based approaches and their forecast combinations with dynamic factor models

In a data-rich environment, forecasting economic variables amounts to extracting and organizing useful information from a large number of predictors. So far, the dynamic factor model and its variants have been the most successful models for such exercises. In this paper, we investigate a category of LASSO-based approaches and evaluate their predictive abilities for forecasting twenty important macroeconomic variables. These alternative models can handle hundreds of data series simultaneously, and extract useful information for forecasting. We also show, both analytically and empirically, that combing forecasts from LASSO-based models with those from dynamic factor models can reduce the mean square forecast error (MSFE) further. Our three main findings can be summarized as follows. First, for most of the variables under investigation, all of the LASSO-based models outperform dynamic factor models in the out-of-sample forecast evaluations. Second, by extracting information and formulating predictors at economically meaningful block levels, the new methods greatly enhance the interpretability of the models. Third, once forecasts from a LASSO-based approach are combined with those from a dynamic factor model by forecast combination techniques, the combined forecasts are significantly better than either dynamic factor model forecasts or the naïve random walk benchmark.     

Distributed ARIMA models for ultra-long time series

Providing forecasts for ultra-long time series plays a vital role in various activities, such as investment decisions, industrial production arrangements, and farm management. This paper develops a novel distributed forecasting framework to tackle the challenges of forecasting ultra-long time series using the industry-standard MapReduce framework. The proposed model combination approach retains the local time dependency. It utilizes a straightforward splitting across samples to facilitate distributed forecasting by combining the local estimators of time series models delivered from worker nodes and minimizing a global loss function. Instead of unrealistically assuming the data generating process (DGP) of an ultra-long time series stays invariant, we only make assumptions on the DGP of subseries spanning shorter time periods. We investigate the performance of the proposed approach with AutoRegressive Integrated Moving Average (ARIMA) models using the real data application as well as numerical simulations. Our approach improves forecasting accuracy and computational efficiency in point forecasts and prediction intervals, especially for longer forecast horizons, compared to directly fitting the whole data with ARIMA models. Moreover, we explore some potential factors that may affect the forecasting performance of our approach.     

Combination of long term and short term forecasts, with application to tourism demand forecasting

Forecast combination is a well-established and well-tested approach for improving the forecasting accuracy. One beneficial strategy is to use constituent forecasts that have diverse information. In this paper we consider the idea of diversity being accomplished by using different time aggregations. For example, we could create a yearly time series from a monthly time series and produce forecasts for both, then combine the forecasts. These forecasts would each be tracking the dynamics of different time scales, and would therefore add diverse types of information. A comparison of several forecast combination methods, performed in the context of this setup, shows that this is indeed a beneficial strategy and generally provides a forecasting performance that is better than the performances of the individual forecasts that are combined.As a case study, we consider the problem of forecasting monthly tourism numbers for inbound tourism to Egypt. Specifically, we consider 33 individual source countries, as well as the aggregate. The novel combination strategy also produces a generally improved forecasting accuracy.     

LoMEF: A framework to produce local explanations for global model time series forecasts

Global forecasting models (GFMs) that are trained across a set of multiple time series have shown superior results in many forecasting competitions and real-world applications compared with univariate forecasting approaches. One aspect of the popularity of statistical forecasting models such as ETS and ARIMA is their relative simplicity and interpretability (in terms of relevant lags, trend, seasonality, and other attributes), while GFMs typically lack interpretability, especially relating to particular time series. This reduces the trust and confidence of stakeholders when making decisions based on the forecasts without being able to understand the predictions. To mitigate this problem, we propose a novel local model-agnostic interpretability approach to explain the forecasts from GFMs. We train simpler univariate surrogate models that are considered interpretable (e.g., ETS) on the predictions of the GFM on samples within a neighbourhood that we obtain through bootstrapping, or straightforwardly as the one-step-ahead global black-box model forecasts of the time series which needs to be explained. After, we evaluate the explanations for the forecasts of the global models in both qualitative and quantitative aspects such as accuracy, fidelity, stability, and comprehensibility, and are able to show the benefits of our approach.     

Simpler Probabilistic Population Forecasts: Making Scenarios Work

The traditional high-low-medium scenario approach to quantifying uncertainty in population forecasts has been criticized as lacking probabilistic meaning and consistency. This paper shows, under certain assumptions, how appropriately calibrated scenarios can be used to approximate the uncertainty intervals on future population size and age structure obtained with fully stochastic forecasts. As many forecasting organizations already produce scenarios and because dealing with them is familiar territory, the methods presented here offer an attractive intermediate position between probabilistically inconsistent scenario analysis and fully stochastic forecasts.     

Applications of machine learning for corporate bond yield spread forecasting

This article considers nine different predictive techniques, including state-of-the-art machine learning methods for forecasting corporate bond yield spreads with other input variables. We examine each method’s out-of-sample forecasting performance using two different forecast horizons: (1) the in-sample dataset over 2003–2007 is used for one-year-ahead and two-year-ahead forecasts of non-callable corporate bond yield spreads; and (2) the in-sample dataset over 2003–2008 is considered to forecast the yield spreads in 2009. Evaluations of forecasting accuracy have shown that neural network forecasts are superior to the other methods considered here in both the short and longer horizon. Furthermore, we visualize the determinants of yield spreads and find that a firm’s equity volatility is a critical factor in yield spreads.     

Data-driven forecasting for operational planning of emergency medical services

Emergency medical services (EMS) play a vital role in delivering pre-hospital care. The operational efficiency of such services is critical and adequate demand forecasts can contribute to such a goal. But for that, the available data need to be well characterized before being used. Previous studies have failed to address some important aspects of this need, such as exploring a comprehensive list of contextual data to decide which are relevant to explain the EMS demand behavior. Moreover, modern forecasting techniques have been explored in the EMS context, including neural networks, but the computational complexity inherent to the methods and their use was not discussed. Finally, it is also unclear how different demand patterns can be when predicting the volume of emergency calls considering the priority level and the number of dispatches according to vehicle type. This study proposes a generic data-driven forecasting method to address these shortcomings and to support operational decisions. The results obtained with the proposed method indicate that each priority call and vehicle type shows different patterns, which suggests that such differentiation should contribute to better resource allocation. At the same time, the operational impact of the demand shared by neighboring zones proved to be significant at bases near the border. The models developed resulted in important decision tools that can be used to predict the dynamic demand of EMS on an hourly or shift basis. Additionally, the method adds value for decision-makers that want to plan not only when and how many but also where resources are demanded, avoiding assumptions that impact the operational performance.     

EVALUATING MACROECONOMIC FORECASTS: A CONCISE REVIEW OF SOME RECENT DEVELOPMENTS

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.     

Influence of differentiated roles on group forecasting accuracy

While behavioral research on forecasting has mostly examined the individual forecaster, organizationally-based forecasting processes typically tend to rely on groups with members from different functional areas for arriving at ‘consensus’ forecasts. The forecasting performance could also vary depending on the particular group structuring utilized in reaching a final prediction. The current study compares the forecasting performance of modified consensus groups with that of staticized groups using formal role-playing. It is found that, when undistorted model forecasts are given, group forecasts (whether they are arrived at through averaging or by a detailed discussion of the forecasts) contribute positively to the forecasting accuracy. However, providing distorted initial forecasts affects the final accuracy with varying degrees of improvement over the initial forecasts. The results show a strong tendency to favor optimistic forecasts for both the staticized and modified consensus group forecasts. Overall, the role modifications are found to be successful in eliciting a differential adjustment behavior, effectively mimicking the disparities between different organizational roles. Current research suggests that group discussions may be an efficient method of displaying and resolving differential motivational contingencies, potentially leading to group forecasts that perform quite well.     

Are GDP forecasts optimal? Evidence on European countries

We assess the accuracy of real GDP growth forecasts released by governments and international organizations for European countries in the years 1999–2017. We implement three testing procedures characterized by different assumptions on the forecasters’ loss functions. First, we test forecast rationality within the traditional approach based on a quadratic loss function (Mincer and Zarnowitz, 1969). Second, following Elliott, Timmermann and Komunjer (2005), we test rationality by allowing for a flexible loss function where the shape parameter driving the extent of asymmetry is unknown and estimated from the empirical distribution of forecast errors. Lastly, we implement the tests proposed by Patton and Timmermann (2007a) that hold regardless of the functional form of the loss function. We conclude that governmental forecasts are biased and not rational under a symmetric and quadratic loss function, but they are optimal under more general assumptions on the loss function. We also find that the preferences of forecasters change with the forecasting horizon: when moving from one- to two-year-ahead forecasts, the optimistic bias increases and the parameter of asymmetry in the loss function significantly increases.     

Density forecasting with Bayesian Vector Autoregressive models under macroeconomic data uncertainty

Macroeconomic data are subject to data revisions. Yet, the usual way of generating real-time density forecasts from Bayesian Vector Autoregressive (BVAR) models makes no allowance for data uncertainty from future data revisions. We develop methods of allowing for data uncertainty when forecasting with BVAR models with stochastic volatility. First, the BVAR forecasting model is estimated on real-time vintages. Second, the BVAR model is jointly estimated with a model of data revisions such that forecasts are conditioned on estimates of the ‘true’ values. We find that this second method generally improves upon conventional practice for density forecasting, especially for the United States.