Forecast combination for U.S. recessions with real-time data

This paper proposes the use of forecast combination to improve predictive accuracy in forecasting the U.S. business cycle index, as published by the Business Cycle Dating Committee of the NBER. It focuses on one-step ahead out-of-sample monthly forecast utilising the well-established coincident indicators and yield curve models, allowing for dynamics and real-time data revisions. Forecast combinations use log-score and quadratic-score based weights, which change over time. This paper finds that forecast accuracy improves when combining the probability forecasts of both the coincident indicators model and the yield curve model, compared to each model's own forecasting performance.     

Real-time squared: A real-time data set for real-time GDP forecasting

This paper uses real-time data to mimic real-time GDP forecasting activity. Through automatic searches for the best indicators for predicting GDP one and four steps ahead, we compare the out-of-sample forecasting performance of adaptive models using different data vintages, and produce three main findings. First, despite data revisions, the forecasting performance of models with indicators is better, but this advantage tends to vanish over longer forecasting horizons. Second, the practice of using fully updated datasets at the time the forecast is made (i.e., taking the best available measures of today's economic situation) does not appear to bring any effective improvement in forecasting ability: the first GDP release is predicted equally well by models using real-time data as by models using the latest available data. Third, although the first release is a rational forecast of GDP data after all statistical revisions have taken place, the forecast based on the latest available GDP data (i.e. the “temporarily best” measures) may be improved by combining preliminary official releases with one-step-ahead forecasts.     

FFORMPP: Feature-based forecast model performance prediction

This paper introduces a novel meta-learning algorithm for time series forecast model performance prediction. We model the forecast error as a function of time series features calculated from historical time series with an efficient Bayesian multivariate surface regression approach. The minimum predicted forecast error is then used to identify an individual model or a combination of models to produce the final forecasts. It is well known that the performance of most meta-learning models depends on the representativeness of the reference dataset used for training. In such circumstances, we augment the reference dataset with a feature-based time series simulation approach, namely GRATIS, to generate a rich and representative time series collection. The proposed framework is tested using the M4 competition data and is compared against commonly used forecasting approaches. Our approach provides comparable performance to other model selection and combination approaches but at a lower computational cost and a higher degree of interpretability, which is important for supporting decisions. We also provide useful insights regarding which forecasting models are expected to work better for particular types of time series, the intrinsic mechanisms of the meta-learners, and how the forecasting performance is affected by various factors.     

Distributed ARIMA models for ultra-long time series

Providing forecasts for ultra-long time series plays a vital role in various activities, such as investment decisions, industrial production arrangements, and farm management. This paper develops a novel distributed forecasting framework to tackle the challenges of forecasting ultra-long time series using the industry-standard MapReduce framework. The proposed model combination approach retains the local time dependency. It utilizes a straightforward splitting across samples to facilitate distributed forecasting by combining the local estimators of time series models delivered from worker nodes and minimizing a global loss function. Instead of unrealistically assuming the data generating process (DGP) of an ultra-long time series stays invariant, we only make assumptions on the DGP of subseries spanning shorter time periods. We investigate the performance of the proposed approach with AutoRegressive Integrated Moving Average (ARIMA) models using the real data application as well as numerical simulations. Our approach improves forecasting accuracy and computational efficiency in point forecasts and prediction intervals, especially for longer forecast horizons, compared to directly fitting the whole data with ARIMA models. Moreover, we explore some potential factors that may affect the forecasting performance of our approach.     

Forecasting Errors: the Track Record

This Briefing Paper is the last of a series of three about forecasting. In this one we examine our forecasting record; it complements the February paper in which we analysed the properties of our forecasting model in terms of the error bands attached to the central forecast.
There are many ways of measuring forecasting errors, and in the first part of this Briefing Paper we describe briefing how we have tackled the problem. (A more detailed analysis can be found in the Appendix.) In Part II we report and comment upon the errors in our forecasts of annual growth rates and show how our forecasting performance has improved over the years. In Part III we focus on quarterly forecasts up to 8 quarters ahead, and compare our forecasting errors with measurement errors in the oficial statistics; with the estimation errors built into our forecast equations; and with the stochastic model errors we reported last February. A brief summary of the main conclusions is given below.     

Comparing forecasts from fixed and variable coefficient models: The case of money demand

In this paper we introduce a class of tentatively plausible, fixed-coefficient models of money demand and evaluate their forecast performance. When these models are reestimated allowing all coefficients to vary over time, the forecasting performance improves dramatically. Aside from offering insights about improved methods of analyzing time series data, the most promising direct use for point estimates derived from time-varying coefficients is as an aid in calibrating proposed models of the kind discussed here.     

Approximate Bayesian forecasting

Approximate Bayesian Computation (ABC) has become increasingly prominent as a method for conducting parameter inference in a range of challenging statistical problems, most notably those characterized by an intractable likelihood function. In this paper, we focus on the use of ABC not as a tool for parametric inference, but as a means of generating probabilistic forecasts; or for conducting what we refer to as ‘approximate Bayesian forecasting’. The four key issues explored are: (i) the link between the theoretical behavior of the ABC posterior and that of the ABC-based predictive; (ii) the use of proper scoring rules to measure the (potential) loss of forecast accuracy when using an approximate rather than an exact predictive; (iii) the performance of approximate Bayesian forecasting in state space models; and (iv) the use of forecasting criteria to inform the selection of ABC summaries in empirical settings. The primary finding of the paper is that ABC can provide a computationally efficient means of generating probabilistic forecasts that are nearly identical to those produced by the exact predictive, and in a fraction of the time required to produce predictions via an exact method.     

DIRECT MULTI-STEP ESTIMATION AND FORECASTING

Abstract.  This paper surveys the literature on multi-step forecasting when the model or the estimation method focuses directly on the link between the forecast origin and the horizon of interest. Among diverse contributions, we show how the current consensual concepts have emerged. We present an exhaustive overview of the existing results, including a conclusive review of the circumstances favourable to direct multi-step forecasting, namely different forms of non-stationarity and appropriate model design. We also provide a unifying framework which allows us to analyse the sources of forecast errors and hence of accuracy improvements from direct over iterated multi-step forecasting.     

A combined random effect and fixed effect forecast for panel data models

When some of the regressors in a panel data model are correlated with the random individual effects, the random effect (RE) estimator becomes inconsistent while the fixed effect (FE) estimator is consistent. Depending on the various degree of such correlation, we can combine the RE estimator and FE estimator to form a combined estimator which can be better than each of the FE and RE estimators. In this paper, we are interested in whether the combined estimator may be used to form a combined forecast to improve upon the RE forecast (forecast made using the RE estimator) and the FE forecast (forecast using the FE estimator) in out-of-sample forecasting. Our simulation experiment shows that the combined forecast does dominate the FE forecast for all degrees of endogeneity in terms of mean squared forecast errors (MSFE), demonstrating that the theoretical results of the risk dominance for the in-sample estimation carry over to the out-of-sample forecasting. It also shows that the combined forecast can reduce MSFE relative to the RE forecast for moderate to large degrees of endogeneity and for large degrees of heterogeneity in individual effects.     

Real-time forecasting of German GDP based on a large factor model with monthly and quarterly data

This paper discusses a factor model for short-term forecasting of GDP growth using a large number of monthly and quarterly time series in real-time. To take into account the different periodicities of the data and missing observations at the end of the sample, the factors are estimated by applying an EM algorithm, combined with a principal components estimator. We discuss some in-sample properties of the estimator in a real-time environment and propose alternative methods for forecasting quarterly GDP with monthly factors. In the empirical application, we use a novel real-time dataset for the German economy. Employing a recursive forecast experiment, we evaluate the forecast accuracy of the factor model with respect to German GDP. Furthermore, we investigate the role of revisions in forecast accuracy and assess the contribution of timely monthly observations to the forecast performance. Finally, we compare the performance of the mixed-frequency model with that of a factor model, based on time-aggregated quarterly data.     

Predicting relative forecasting performance: An empirical investigation

The relative performances of forecasting models change over time. This empirical observation raises two questions. First, is the relative performance itself predictable? Second, if so, can it be exploited in order to improve the forecast accuracy? We address these questions by evaluating the predictive abilities of a wide range of economic variables for two key US macroeconomic aggregates, namely industrial production and inflation, relative to simple benchmarks. We find that business cycle indicators, financial conditions, uncertainty and measures of past relative performances are generally useful for explaining the models’ relative forecasting performances. In addition, we conduct a pseudo-real-time forecasting exercise, where we use the information about the conditional performance for model selection and model averaging. The newly proposed strategies deliver sizable improvements over competitive benchmark models and commonly-used combination schemes. The gains are larger when model selection and averaging are based on both financial conditions and past performances measured at the forecast origin date.     

Applications of machine learning for corporate bond yield spread forecasting

This article considers nine different predictive techniques, including state-of-the-art machine learning methods for forecasting corporate bond yield spreads with other input variables. We examine each method’s out-of-sample forecasting performance using two different forecast horizons: (1) the in-sample dataset over 2003–2007 is used for one-year-ahead and two-year-ahead forecasts of non-callable corporate bond yield spreads; and (2) the in-sample dataset over 2003–2008 is considered to forecast the yield spreads in 2009. Evaluations of forecasting accuracy have shown that neural network forecasts are superior to the other methods considered here in both the short and longer horizon. Furthermore, we visualize the determinants of yield spreads and find that a firm’s equity volatility is a critical factor in yield spreads.     

Forecasting and loss functions

This paper considers two problems of interpreting forecasting competition error statistics. The first problem is concerned with the importance of linking the error measure (loss function) used in evaluating a forecasting model with the loss function used in estimating the model. It is argued that because the variety of uses of any single forecast, such matching is impractical. Secondly, there is little evidence that matching would have any impact on comparative forecast performance, however measured. As a consequence the results of forecasting competitions are not affected by this problem. The second problem is concerned with the interpreting performance, when evaluated through M(ean) S(quare) E(rror). The authors show that in the Makridakis Competition, good MSE performance is solely due to performance on a small number of the 1001 series, and arises because of the effects of scale. They conclude that comparisons of forecasting accuracy based on MSE are subject to major problems of interpretation.     

Seasonal exponential smoothing with damped trends : An application for production planning

This paper reviews a spreadsheet-based forecasting approach which a process industry manufacturer developed and implemented to link annual corporate forecasts with its manufacturing/distribution operations. First, we consider how this forecasting system supports overall production planning and why it must be compatible with corporate forecasts. We then review the results of substantial testing of variations on the Winters three-parameter exponential smoothing model on 28 actual product family time series. In particular, we evaluate whether the use of damping parameters improves forecast accuracy. The paper concludes that a Winters four-parameter model (i.e. the standard Winters three-parameter model augmented by a fourth parameter to damp the trend) provides the most accurate forecasts of the models evaluated. Our application confirms the fact that there are situations where the use of damped trend parameters in short-run exponential smoothing based forecasting models is beneficial.     

Macroeconomic forecast accuracy in a data‐rich environment

The performance of six classes of models in forecasting different types of economic series is evaluated in an extensive pseudo out‐of‐sample exercise. One of these forecasting models, regularized data‐rich model averaging (RDRMA), is new in the literature. The findings can be summarized in four points. First, RDRMA is difficult to beat in general and generates the best forecasts for real variables. This performance is attributed to the combination of regularization and model averaging, and it confirms that a smart handling of large data sets can lead to substantial improvements over univariate approaches. Second, the ARMA(1,1) model emerges as the best to forecast inflation changes in the short run, while RDRMA dominates at longer horizons. Third, the returns on the S&P 500 index are predictable by RDRMA at short horizons. Finally, the forecast accuracy and the optimal structure of the forecasting equations are quite unstable over time.     

Sparse estimation of dynamic principal components for forecasting high-dimensional time series

We present the sparse estimation of one-sided dynamic principal components (ODPCs) to forecast high-dimensional time series. The forecast can be made directly with the ODPCs or by using them as estimates of the factors in a generalized dynamic factor model. It is shown that a large reduction in the number of parameters estimated for the ODPCs can be achieved without affecting their forecasting performance.     

Spotting the Danger Zone: Forecasting Financial Crises With Classification Tree Ensembles and Many Predictors

This paper introduces classification tree ensembles (CTEs) to the banking crisis forecasting literature. I show that CTEs substantially improve out‐of‐sample forecasting performance over best‐practice early‐warning systems. CTEs enable policymakers to correctly forecast 80% of crises with a 20% probability of incorrectly forecasting a crisis. These findings are based on a long‐run sample (1870–2011), and two broad post‐1970 samples which together cover almost all known systemic banking crises. I show that the marked improvement in forecasting performance results from the combination of many classification trees into an ensemble, and the use of many predictors. Copyright © 2016 John Wiley & Sons, Ltd.     

Forecasting inflation using time‐varying Bayesian model averaging

This paper presents a Bayesian model averaging regression framework for forecasting US inflation, in which the set of predictors included in the model is automatically selected from a large pool of potential predictors and the set of regressors is allowed to change over time. Using real‐time data on the 1960–2011 period, this model is applied to forecast personal consumption expenditures and gross domestic product deflator inflation. The results of this forecasting exercise show that, although it is not able to beat a simple random‐walk model in terms of point forecasts, it does produce superior density forecasts compared with a range of alternative forecasting models. Moreover, a sensitivity analysis shows that the forecasting results are relatively insensitive to prior choices and the forecasting performance is not affected by the inclusion of a very large set of potential predictors.     

Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition

In this work we introduce the forecasting model with which we participated in the NN5 forecasting competition (the forecasting of 111 time series representing daily cash withdrawal amounts at ATM machines). The main idea of this model is to utilize the concept of forecast combination, which has proven to be an effective methodology in the forecasting literature. In the proposed system we attempted to follow a principled approach, and make use of some of the guidelines and concepts that are known in the forecasting literature to lead to superior performance. For example, we considered various previous comparison studies and time series competitions as guidance in determining which individual forecasting models to test (for possible inclusion in the forecast combination system). The final model ended up consisting of neural networks, Gaussian process regression, and linear models, combined by simple average. We also paid extra attention to the seasonality aspect, decomposing the seasonality into weekly (which is the strongest one), day of the month, and month of the year seasonality.     

Multi-step forecasting in emerging economies: An investigation of the South African GDP

To forecast at several, say h, periods into the future, a modeller faces a choice between iterating one-step-ahead forecasts (the IMS technique), or directly modeling the relationship between observations separated by an h-period interval and using it for forecasting (DMS forecasting). It is known that structural breaks, unit-root non-stationarity and residual autocorrelation may improve DMS accuracy in finite samples, all of which occur when modelling the South African GDP over the period 1965–2000. This paper analyzes the forecasting properties of 779 multivariate and univariate models that combine different techniques of robust forecasting. We find strong evidence supporting the use of DMS and intercept correction, and attribute their superior forecasting performance to their robustness in the presence of breaks.