Medium term growth forecasts: Experts vs. simple models

We compare the medium-term GDP growth forecasts generated by experts to those generated by simple models. This study analyzes a large set of forecasts that covers 48 countries from 1997 to 2016. Out-of-sample exercises indicate that no noticeable difference in performance is observed for advanced economies. In contrast, in the case of emerging economies, model forecasts perform better than expert forecasts. In addition, similar patterns are found for a collection of forecasts from a different set of experts, which suggests that the reported regularity is prevalent. Further analyses suggest that the documented difference in performance can be explained by an optimism bias, excessive reactions to innovations in growth trajectories, and insufficient responses to the information contained in the current account balance.     

Incorporating a tracking signal into a state space model

It is a common practice to complement a forecasting method such as simple exponential smoothing with a monitoring scheme to detect those situations where forecasts have failed to adapt to structural change. It will be suggested in this paper that the equations for simple exponential smoothing can be augmented by a common monitoring statistic to provide a method that automatically adapts to structural change without human intervention. The resulting method, which turns out to be a restricted form of damped trend corrected exponential smoothing, is compared with related methods on the annual data from the M3 competition. It is shown to be better than simple exponential smoothing and more consistent than traditional damped trend exponential smoothing.     

Online hierarchical forecasting for power consumption data

This paper proposes a three-step approach to forecasting time series of electricity consumption at different levels of household aggregation. These series are linked by hierarchical constraints—global consumption is the sum of regional consumption, for example. First, benchmark forecasts are generated for all series using generalized additive models. Second, for each series, the aggregation algorithm ML-Poly, introduced by Gaillard, Stoltz, and van Erven in 2014, finds an optimal linear combination of the benchmarks. Finally, the forecasts are projected onto a coherent subspace to ensure that the final forecasts satisfy the hierarchical constraints. By minimizing a regret criterion, we show that the aggregation and projection steps improve the root mean square error of the forecasts. Our approach is tested on household electricity consumption data; experimental results suggest that successive aggregation and projection steps improve the benchmark forecasts at different levels of household aggregation.     

Forecasting bulk prices of Bordeaux wines using leading indicators

Agricultural price forecasting has been being abandoned progressively by researchers ever since the development of large-scale agricultural futures markets. However, as with many other agricultural goods, there is no futures market for wine. This paper draws on the agricultural prices forecasting literature to develop a forecasting model for bulk wine prices. The price data include annual and monthly series for various wine types that are produced in the Bordeaux region. The predictors include several leading economic indicators of supply and demand shifts. The stock levels and quantities produced are found to have the highest predictive power. The preferred annual and monthly forecasting models outperform naive random walk forecasts by 27.1% and 3.4% respectively; their mean absolute percentage errors are 2.7% and 3.4% respectively. A simple trading strategy based on monthly forecasts is estimated to increase profits by 3.3% relative to a blind strategy that consists of always selling at the spot price.     

Forecasting monthly and quarterly time series using STL decomposition

This paper is a re-examination of the benefits and limitations of decomposition and combination techniques in the area of forecasting, and also a contribution to the field, offering a new forecasting method. The new method is based on the disaggregation of time series components through the STL decomposition procedure, the extrapolation of linear combinations of the disaggregated sub-series, and the reaggregation of the extrapolations to obtain estimates for the global series. Applying the forecasting method to data from the NN3 and M1 Competition series, the results suggest that it can perform well relative to four other standard statistical techniques from the literature, namely the ARIMA, Theta, Holt-Winters’ and Holt’s Damped Trend methods. The relative advantages of the new method are then investigated further relative to a simple combination of the four statistical methods and a Classical Decomposition forecasting method. The strength of the method lies in its ability to predict long lead times with relatively high levels of accuracy, and to perform consistently well for a wide range of time series, irrespective of the characteristics, underlying structure and level of noise of the data.     

Improving forecasting by estimating time series structural components across multiple frequencies

Identifying the most appropriate time series model to achieve a good forecasting accuracy is a challenging task. We propose a novel algorithm that aims to mitigate the importance of model selection, while increasing the accuracy. Multiple time series are constructed from the original time series, using temporal aggregation. These derivative series highlight different aspects of the original data, as temporal aggregation helps in strengthening or attenuating the signals of different time series components. In each series, the appropriate exponential smoothing method is fitted and its respective time series components are forecast. Subsequently, the time series components from each aggregation level are combined, then used to construct the final forecast. This approach achieves a better estimation of the different time series components, through temporal aggregation, and reduces the importance of model selection through forecast combination. An empirical evaluation of the proposed framework demonstrates significant improvements in forecasting accuracy, especially for long-term forecasts.     

Multivariate semi-nonparametric distributions with dynamic conditional correlations

This paper generalizes the Dynamic Conditional Correlation (DCC) model of Engle (2002), incorporating a flexible non-Gaussian distribution based on Gram-Charlier expansions. The resulting semi-nonparametric-DCC (SNP-DCC) model allows estimation in two stages and deals with the negativity problem which is inherent in truncated SNP densities. We test the performance of a SNP-DCC model with respect to the (Gaussian)-DCC through an empirical application of density forecasting for portfolio returns. Our results show that the proposed multivariate model provides a better in-sample fit and forecast of the portfolio returns distribution, and thus is useful for financial risk forecasting and evaluation.     

Expert opinion versus expertise in forecasting

Expert opinion is an opinion given by an expert, and it can have significant value in forecasting key policy variables in economics and finance. Expert forecasts can either be expert opinions, or forecasts based on an econometric model. An expert forecast that is based on an econometric model is replicable, and can be defined as a replicable expert forecast (REF), whereas an expert opinion that is not based on an econometric model can be defined as a non-replicable expert forecast (Non-REF). Both REF and Non-REF may be made available by an expert regarding a policy variable of interest. In this paper, we develop a model to generate REF, and compare REF with Non-REF. A method is presented to compare REF and Non-REF using efficient estimation methods, and a direct test of expertise on expert opinion is given. The latter serves the purpose of investigating whether expert adjustment improves the model-based forecasts. Illustrations for forecasting pharmaceutical stock keeping unit (SKUs), where the econometric model is of (variations of) the autoregressive integrated moving average model (ARIMA) type, show the relevance of the new methodology proposed in the paper. In particular, experts possess significant expertise, and expert forecasts are significant in explaining actual sales.     

Tracking world trade and GDP in real time

This paper proposes a simple procedure for obtaining monthly assessments of short-run perspectives for quarterly world GDP and trade. It combines high-frequency information from emerging and advanced countries so as to explain quarterly national accounts variables through bridge models. The union of all bridge equations leads to our world bridge model (WBM). The WBM approach of this paper is new for two reasons: its equations combine traditional short-run bridging with theoretical level-relationships, and it is the first time that forecasts of world GDP and trade have been computed for both advanced and emerging countries on the basis of a real-time database of approximately 7000 time series. Although the performances of the equations that are searched automatically should be taken as a lower bound, our results show that the forecasting ability of the WBM is superior to the benchmark. Finally, our results confirm that the use of revised data leads to models’ forecasting performances being overstated significantly.     

Evaluating quantile forecasts in the M5 uncertainty competition

Probabilistic forecasts are necessary for robust decisions in the face of uncertainty. The M5 Uncertainty competition required participating teams to forecast nine quantiles for unit sales of various products at various aggregation levels and for different time horizons. This paper evaluates the forecasting performance of the quantile forecasts at different aggregation levels and at different quantile levels. We contrast this with some theoretical predictions, and discuss potential implications and promising future research directions for the practice of probabilistic forecasting.     

Improving the wisdom of crowds with analysis of variance of predictions of related outcomes

Decision-makers often collect and aggregate experts’ point predictions about continuous outcomes, such as stock returns or product sales. In this article, we model experts as Bayesian agents and show that means, including the (weighted) arithmetic mean, trimmed means, median, geometric mean, and essentially all other measures of central tendency, do not use all information in the predictions. Intuitively, they assume idiosyncratic differences to arise from error instead of private information and hence do not update the prior with all available information. Updating means in terms of unused information improves their expected accuracy but depends on the experts’ prior and information structure that cannot be estimated based on a single prediction per expert. In many applications, however, experts consider multiple stocks, products, or other related items at the same time. For such contexts, we introduce ANOVA updating – an unsupervised technique that updates means based on experts’ predictions of multiple outcomes from a common population. The technique is illustrated on several real-world datasets.     

Combining survey long-run forecasts and nowcasts with BVAR forecasts using relative entropy

This paper constructs hybrid forecasts that combine forecasts from vector autoregressive (VAR) model(s) with both short- and long-term expectations from surveys. Specifically, we use the relative entropy to tilt one-step-ahead and long-horizon VAR forecasts to match the nowcasts and long-horizon forecasts from the Survey of Professional Forecasters. We consider a variety of VAR models, ranging from simple fixed-parameter to time-varying parameters. The results across models indicate meaningful gains in multi-horizon forecast accuracy relative to model forecasts that do not incorporate long-term survey conditions. Accuracy improvements are achieved for a range of variables, including those that are not tilted directly but are affected through spillover effects from tilted variables. The accuracy gains for hybrid inflation forecasts from simple VARs are substantial, statistically significant, and competitive to time-varying VARs, univariate benchmarks, and survey forecasts. We view our proposal as an indirect approach to accommodating structural change and moving end points.     

Forecasting temperature to price CME temperature derivatives

This paper seeks to forecast temperatures in US cities in order to price temperature derivatives on the Chicago Mercantile Exchange (CME). The CME defines the average daily temperature underlying its contracts as the average of the maximum and minimum daily temperatures, yet all published work on temperature forecasting for pricing purposes has ignored this peculiar definition of the average and sought to model the average temperature directly. This paper is the first to look at the average temperature forecasting problem as an analysis of extreme values. The theory of extreme values guides model selection for temperature maxima and minima, and a forecast distribution for the CME’s daily average temperature is found through convolution. While univariate time series AR-GARCH and regression models generally yield superior point forecasts of temperatures, our extreme-value-based model consistently outperforms these models in density forecasting, the most important risk management tool.     

On testing weak separability

A new framework for testing for the existence of a consistent aggregator for a subset of inputs in a production function is developed in terms of the variable profit function. In contrast to the Berndt–Christensen framework, in which the parametric restrictions required to attain weak separability also impose unwanted restrictions on the form of the aggregator, the aggregator function has a flexible functional form. Consequently this procedure should permit a less restrictive test of separability or aggregation. Application of the procedure to the data for U.S. manufacturing assuming a production function involving two labour inputs (blue- and white-collar workers) and two capital inputs (structures and equipment) leads to the conclusion that there does not exist a consistent aggregator for labor whereas there is some mild support for the existence of a consistent aggregator for capital.     

EVALUATING MACROECONOMIC FORECASTS: A CONCISE REVIEW OF SOME RECENT DEVELOPMENTS

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.     

Forecasting with vector autoregressive (VAR) models subject to business cycle restrictions

In the last decade VAR models have become a widely-used tool for forecasting macroeconomic time series. To improve the out-of-sample forecasting accuracy of these models, Bayesian random-walk prior restrictions are often imposed on VAR model parameters. This paper focuses on whether placing an alternative type of restriction on the parameters of unrestricted VAR models improves the out-of-sample forecasting performance of these models. The type of restriction analyzed here is based on the business cycle characteristics of U.S. macroeconomic data, and in particular, requires that the dynamic behavior of the restricted VAR model mimic the business cycle characteristics of historical data. The question posed in this paper is: would a VAR model, estimated subject to the restriction that the cyclical characteristics of simulated data from the model “match up” with the business cycle characteristics of U.S. data, generate more accurate out-of-sample forecasts than unrestricted or Bayesian VAR models?     

Predicting relative forecasting performance: An empirical investigation

The relative performances of forecasting models change over time. This empirical observation raises two questions. First, is the relative performance itself predictable? Second, if so, can it be exploited in order to improve the forecast accuracy? We address these questions by evaluating the predictive abilities of a wide range of economic variables for two key US macroeconomic aggregates, namely industrial production and inflation, relative to simple benchmarks. We find that business cycle indicators, financial conditions, uncertainty and measures of past relative performances are generally useful for explaining the models’ relative forecasting performances. In addition, we conduct a pseudo-real-time forecasting exercise, where we use the information about the conditional performance for model selection and model averaging. The newly proposed strategies deliver sizable improvements over competitive benchmark models and commonly-used combination schemes. The gains are larger when model selection and averaging are based on both financial conditions and past performances measured at the forecast origin date.     

Fast and accurate yearly time series forecasting with forecast combinations

It has long been known that combination forecasting strategies produce superior out-of-sample forecasting performances. In the M4 forecasting competition, a very simple forecast combination strategy achieved third place on yearly time series. An analysis of the ensemble model and its component models suggests that the competitive accuracy comes from avoiding poor forecasts, rather than from beating the best individual models. Moreover, the simple ensemble model can be fitted very quickly, can easily scale horizontally with additional CPU cores or a cluster of computers, and can be implemented by users very quickly and easily. This approach might be of particular interest to users who need accurate yearly forecasts without being able to spend significant time, resources, or expertise on tuning models. Users of the R statistical programming language can access this modeling approach using the “forecastHybrid” package.     

A hybrid system-identification method for forecasting telecommunications product demands

A crucial challenge for telecommunications companies is how to forecast changes in demand for specific products over the next 6 to 18 months—the length of a typical short-range capacity-planning and capital-budgeting planning horizon. The problem is especially acute when only short histories of product sales are available. This paper presents a new two-level approach to forecasting demand from short-term data. The lower of the two levels consists of adaptive system-identification algorithms borrowed from signal processing, especially, Hidden Markov Model (HMM) methods [Hidden Markov Models: Estimation and Control (1995) Springer Verlag]. Although they have primarily been used in engineering applications such as automated speech recognition and seismic data processing, HMM techniques also appear to be very promising for predicting probabilities of individual customer behaviors from relatively short samples of recent product-purchasing histories. The upper level of our approach applies a classification tree algorithm to combine information from the lower-level forecasting algorithms. In contrast to other forecast-combination algorithms, such as weighted averaging or Bayesian aggregation formulas, the classification tree approach exploits high-order interactions among error patterns from different predictive systems. It creates a hybrid, forecasting algorithm that out-performs any of the individual algorithms on which it is based. This tree-based approach to hybridizing forecasts provides a new, general way to combine and improve individual forecasts, whether or not they are based on HMM algorithms. The paper concludes with the results of validation tests. These show the power of HMM methods to forecast what individual customers are likely to do next. They also show the gain from classification tree post-processing of the predictions from lower-level forecasts. In essence, these techniques enhance the limited techniques available for new product forecasting.     

Content aggregation by platforms: The case of the news media

The digitization of content has led to the emergence of platforms that draw information from multiple sources. This paper investigates whether aggregation of content by a single platform encourages users to “skim” content or to investigate it in depth. We study a contract dispute that led a major aggregator to remove information from a major content provider. After the removal, users were less likely to investigate additional, related content in depth, particularly sources that were horizontally or vertically differentiated.