Combining VAR and DSGE forecast densities

A popular macroeconomic forecasting strategy utilizes many models to hedge against instabilities of unknown timing; see (among others) Stock and Watson (2004), Clark and McCracken (2010), and Jore et al. (2010). Existing studies of this forecasting strategy exclude dynamic stochastic general equilibrium (DSGE) models, despite the widespread use of these models by monetary policymakers. In this paper, we use the linear opinion pool to combine inflation forecast densities from many vector autoregressions (VARs) and a policymaking DSGE model. The DSGE receives a substantial weight in the pool (at short horizons) provided the VAR components exclude structural breaks. In this case, the inflation forecast densities exhibit calibration failure. Allowing for structural breaks in the VARs reduces the weight on the DSGE considerably, but produces well-calibrated forecast densities for inflation.     

Factor MIDAS for Nowcasting and Forecasting with Ragged‐Edge Data: A Model Comparison for German GDP*

In this article, we merge two strands from the recent econometric literature. First, factor models based on large sets of macroeconomic variables for forecasting, which have generally proven useful for forecasting. However, there is some disagreement in the literature as to the appropriate method. Second, forecast methods based on mixed‐frequency data sampling (MIDAS). This regression technique can take into account unbalanced datasets that emerge from publication lags of high‐ and low‐frequency indicators, a problem practitioner have to cope with in real time. In this article, we introduce Factor MIDAS, an approach for nowcasting and forecasting low‐frequency variables like gross domestic product (GDP) exploiting information in a large set of higher‐frequency indicators. We consider three alternative MIDAS approaches (basic, smoothed and unrestricted) that provide harmonized projection methods that allow for a comparison of the alternative factor estimation methods with respect to nowcasting and forecasting. Common to all the factor estimation methods employed here is that they can handle unbalanced datasets, as typically faced in real‐time forecast applications owing to publication lags. In particular, we focus on variants of static and dynamic principal components as well as Kalman filter estimates in state‐space factor models. As an empirical illustration of the technique, we use a large monthly dataset of the German economy to nowcast and forecast quarterly GDP growth. We find that the factor estimation methods do not differ substantially, whereas the most parsimonious MIDAS projection performs best overall. Finally, quarterly models are in general outperformed by the Factor MIDAS models, which confirms the usefulness of the mixed‐frequency techniques that can exploit timely information from business cycle indicators.     

Sectoral Survey‐based Confidence Indicators for Europe*

We analyse a novel dataset of Business and Consumer Surveys, using dynamic factor techniques, to produce composite coincident indices (CCIs) at the sectoral level for the European countries and for Europe. Surveys are timely available, not subject to revision, and fully comparable across countries. Moreover, the substantial discrepancies in activity at the sectoral level justify the interest in sectoral disaggregation. Compared with the confidence indicators produced by the European Commission we show that factor‐based CCIs, using survey answers at a more disaggregate level, produce higher correlation with the reference series for the majority of sectors and countries.     

Leading Indicators for Euro‐area Inflation and GDP Growth*

In this paper, we evaluate the role of a set of variables as leading indicators for Euro‐area inflation and GDP growth. Our leading indicators are taken from the variables in the European Central Bank's (ECB) Euro‐area‐wide model database, plus a set of similar variables for the US. We compare the forecasting performance of each indicator ex post with that of purely autoregressive models. We also analyse three different approaches to combining the information from several indicators. First, ex post, we discuss the use as indicators of the estimated factors from a dynamic factor model for all the indicators. Secondly, within an ex ante framework, an automated model selection procedure is applied to models with a large set of indicators. No future information is used, future values of the regressors are forecast, and the choice of the indicators is based on their past forecasting records. Finally, we consider the forecasting performance of groups of indicators and factors and methods of pooling the ex ante single‐indicator or factor‐based forecasts. Some sensitivity analyses are also undertaken for different forecasting horizons and weighting schemes of forecasts to assess the robustness of the results.     

Evaluating,Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation*

This paper proposes and analyses the Kullback–Leibler information criterion (KLIC) as a unified statistical tool to evaluate, compare and combine density forecasts. Use of the KLIC is particularly attractive, as well as operationally convenient, given its equivalence with the widely used Berkowitz likelihood ratio test for the evaluation of individual density forecasts that exploits the probability integral transforms. Parallels with the comparison and combination of point forecasts are made. This and related Monte Carlo experiments help draw out properties of combined density forecasts. We illustrate the uses of the KLIC in an application to two widely used published density forecasts for UK inflation, namely the Bank of England and NIESR ‘fan’ charts.     

Combining Density and Interval Forecasts: A Modest Proposal*

The finite mixture distribution is proposed as an appropriate statistical model for a combined density forecast. Its implications for measures of uncertainty and disagreement, and for combining interval forecasts, are described. Related proposals in the literature and applications to the U.S. Survey of Professional Forecasters are discussed.     

Transforming Qualitative Survey Data: Performance Comparisons for the UK*

This paper assesses different ways of converting qualitative data obtained in surveys into quantitative indices for a number of economic variables. The research reported here focuses on the main UK employers’ business survey for manufacturing – the CBI industrial trends survey. Six response variables are investigated – plant and machinery investment, output, employment, exports, price and cost. We find that the balance statistic is a satisfactory method of transforming three of the variables: investment, output and exports.     

A Simple Explanation of the Forecast Combination Puzzle*

This article presents a formal explanation of the forecast combination puzzle, that simple combinations of point forecasts are repeatedly found to outperform sophisticated weighted combinations in empirical applications. The explanation lies in the effect of finite‐sample error in estimating the combining weights. A small Monte Carlo study and a reappraisal of an empirical study by Stock and Watson [Federal Reserve Bank of Richmond Economic Quarterly (2003) Vol. 89/3, pp. 71–90] support this explanation. The Monte Carlo evidence, together with a large‐sample approximation to the variance of the combining weight, also supports the popular recommendation to ignore forecast error covariances in estimating the weight.     

New Methods for Forecasting Inflation,Applied to the US*

Models for the 12‐month‐ahead US rate of inflation, measured by the chain‐weighted consumer expenditure deflator, are estimated for 1974–98 and subsequent pseudo out‐of‐sample forecasting performance is examined. Alternative forecasting approaches for different information sets are compared with benchmark univariate autoregressive models, and substantial out‐performance is demonstrated including against Stock and Watson's unobserved components‐stochastic volatility model. Three key ingredients to the out‐performance are: including equilibrium correction component terms in relative prices; introducing nonlinearities to proxy state‐dependence in the inflation process and replacing the information criterion, commonly used in VARs to select lag length, with a ‘parsimonious longer lags’ parameterization. Forecast pooling or averaging also improves forecast performance.     

Time-varying combinations of predictive densities using nonlinear filtering

We propose a Bayesian combination approach for multivariate predictive densities which relies upon a distributional state space representation of the combination weights. Several specifications of multivariate time-varying weights are introduced with a particular focus on weight dynamics driven by the past performance of the predictive densities and the use of learning mechanisms. In the proposed approach the model set can be incomplete, meaning that all models can be individually misspecified. A Sequential Monte Carlo method is proposed to approximate the filtering and predictive densities. The combination approach is assessed using statistical and utility-based performance measures for evaluating density forecasts of simulated data, US macroeconomic time series and surveys of stock market prices. Simulation results indicate that, for a set of linear autoregressive models, the combination strategy is successful in selecting, with probability close to one, the true model when the model set is complete and it is able to detect parameter instability when the model set includes the true model that has generated subsamples of data. Also, substantial uncertainty appears in the weights when predictors are similar; residual uncertainty reduces when the model set is complete; and learning reduces this uncertainty. For the macro series we find that incompleteness of the models is relatively large in the 1970’s, the beginning of the 1980’s and during the recent financial crisis, and lower during the Great Moderation; the predicted probabilities of recession accurately compare with the NBER business cycle dating; model weights have substantial uncertainty attached. With respect to returns of the S&P 500 series, we find that an investment strategy using a combination of predictions from professional forecasters and from a white noise model puts more weight on the white noise model in the beginning of the 1990’s and switches to giving more weight to the professional forecasts over time. Information on the complete predictive distribution and not just on some moments turns out to be very important, above all during turbulent times such as the recent financial crisis. More generally, the proposed distributional state space representation offers great flexibility in combining densities.     

Evolution of forecast disagreement in a Bayesian learning model

We estimate a Bayesian learning model with heterogeneity aimed at explaining expert forecast disagreement and its evolution over horizons. Disagreement is postulated to have three components due to differences in: (i) the initial prior beliefs, (ii) the weights attached on priors, and (iii) interpreting public information. The fixed-target, multi-horizon, cross-country feature of the panel data allows us to estimate the relative importance of each component precisely. The first component explains nearly all to 30% of forecast disagreement as the horizon decreases from 24 months to 1 month. This finding firmly establishes the role of initial prior beliefs in generating expectation stickiness. We find the second component to have barely any effect on the evolution of forecast disagreement among experts. The importance of the third component increases from almost nothing to 70% as the horizon gets shorter via its interaction with the quality of the incoming news. We propose a new test of forecast efficiency in the context of Bayesian information processing and find significant heterogeneity in the nature of inefficiency across horizons and countries.     

Model uncertainty and policy evaluation: Some theory and empirics

This paper explores ways to integrate model uncertainty into policy evaluation. We describe a general framework that includes both model averaging methods as well as some measures that describe whether policies and their consequences are model dependent. These general ideas are then applied to assess simple monetary policy rules for some standard New Keynesian specifications. We conclude that the original Taylor rule has good robustness properties, but may reasonably be challenged in overall quality with respect to stabilization by alternative simple rules, even when these rules employ parameters that are set without accounting for model uncertainty.     

Likelihood-based scoring rules for comparing density forecasts in tails

We propose new scoring rules based on conditional and censored likelihood for assessing the predictive accuracy of competing density forecasts over a specific region of interest, such as the left tail in financial risk management. These scoring rules can be interpreted in terms of Kullback-Leibler divergence between weighted versions of the density forecast and the true density. Existing scoring rules based on weighted likelihood favor density forecasts with more probability mass in the given region, rendering predictive accuracy tests biased toward such densities. Using our novel likelihood-based scoring rules avoids this problem.     

Efficient information theoretic inference for conditional moment restrictions

The generalised method of moments estimator may be substantially biased in finite samples, especially so when there are large numbers of unconditional moment conditions. This paper develops a class of first-order equivalent semi-parametric efficient estimators and tests for conditional moment restrictions models based on a local or kernel-weighted version of the Cressie–Read power divergence family of discrepancies. This approach is similar in spirit to the empirical likelihood methods of Kitamura et al. [2004. Empirical likelihood-based inference in conditional moment restrictions models. Econometrica 72, 1667–1714] and Tripathi and Kitamura [2003. Testing conditional moment restrictions. Annals of Statistics 31, 2059–2095]. These efficient local methods avoid the necessity of explicit estimation of the conditional Jacobian and variance matrices of the conditional moment restrictions and provide empirical conditional probabilities for the observations.     

Nonparametric tests for conditional symmetry in dynamic models

This article proposes omnibus tests for conditional symmetry around a parametric function in a dynamic context. Conditional moments may not exist or may depend on the explanatory variables. Test statistics are suitable functionals of the empirical process of residuals and explanatory variables, whose limiting distribution under the null is nonpivotal. The tests are implemented with the assistance of a bootstrap method, which is justified assuming very mild regularity conditions on the specification of the center of symmetry and the underlying serial dependence structure. Finite sample properties are examined by means of a Monte Carlo experiment.     

Least-squares forecast averaging

This paper proposes forecast combination based on the method of Mallows Model Averaging (MMA). The method selects forecast weights by minimizing a Mallows criterion. This criterion is an asymptotically unbiased estimate of both the in-sample mean-squared error (MSE) and the out-of-sample one-step-ahead mean-squared forecast error (MSFE). Furthermore, the MMA weights are asymptotically mean-square optimal in the absence of time-series dependence. We show how to compute MMA weights in forecasting settings, and investigate the performance of the method in simple but illustrative simulation environments. We find that the MMA forecasts have low MSFE and have much lower maximum regret than other feasible forecasting methods, including equal weighting, BIC selection, weighted BIC, AIC selection, weighted AIC, Bates–Granger combination, predictive least squares, and Granger–Ramanathan combination.     

Optimal forecasts in the presence of structural breaks

This paper considers the problem of forecasting under continuous and discrete structural breaks and proposes weighting observations to obtain optimal forecasts in the MSFE sense. We derive optimal weights for one step ahead forecasts. Under continuous breaks, our approach largely recovers exponential smoothing weights. Under discrete breaks, we provide analytical expressions for optimal weights in models with a single regressor, and asymptotically valid weights for models with more than one regressor. It is shown that in these cases the optimal weight is the same across observations within a given regime and differs only across regimes. In practice, where information on structural breaks is uncertain, a forecasting procedure based on robust optimal weights is proposed. The relative performance of our proposed approach is investigated using Monte Carlo experiments and an empirical application to forecasting real GDP using the yield curve across nine industrial economies.     

Finance-neutral potential output: An evaluation in an emerging market monetary policy context

In this paper, output gaps that include financial cycle information are evaluated against policy analysis models used by the Colombian central bank. This is an important feature, since policy-related models are the only relevant yardstick and emerging economies (such as Colombia) have been historically more vulnerable to financial imbalances. Unlike previous works, finance-neutral gaps were evaluated in a monetary policy context exactly as it is routinely performed by a central bank. The distribution of output gap revisions is analyzed and a metric to compare real-time robustness across models is developed. This metric constitutes a novel way to summarize the distribution of real-time uncertainty around output gaps, and policymakers should employ it for comparison purposes. Also, the real-time policy performance of finance-neutral gaps is studied, separating suggested ex post from operational ex ante usefulness. The results suggest that finance-neutral gaps are neither more robust in real time nor more operationally useful than the benchmark estimates. These results have important implications for policymakers and for the relevant literature.     

Goodness-of-fit tests for conditional models under censoring and truncation

The problem of specification tests for conditional models is studied when the data are subject to left truncation and right censoring. A general method is applied to derive tests for the polynomial regression, the proportional hazards, the additive risks and the proportional odds models. Bootstrap versions to approximate the critical values of the test are introduced and proved to work both from a theoretical viewpoint as well as in a small simulation study.     

Out of sample forecasts of quadratic variation

We compare the forecasts of Quadratic Variation given by the Realized Volatility (RV) and the Two Scales Realized Volatility (TSRV) computed from high frequency data in the presence of market microstructure noise, under several different dynamics for the volatility process and assumptions on the noise. We show that TSRV largely outperforms RV, whether looking at bias, variance, RMSE or out-of-sample forecasting ability. An empirical application to all DJIA stocks confirms the simulation results.