A comprehensive evaluation of macroeconomic forecasting methods

We employ datasets for seven developed economies and consider four classes of multivariate forecasting models in order to extend and enhance the empirical evidence in the macroeconomic forecasting literature. The evaluation considers forecasting horizons of between one quarter and two years ahead. We find that the structural model, a medium-sized DSGE model, provides accurate long-horizon US and UK inflation forecasts. We strike a balance between being comprehensive and producing clear messages by applying meta-analysis regressions to 2,976 relative accuracy comparisons that vary with the forecasting horizon, country, model class and specification, number of predictors, and evaluation period. For point and density forecasting of GDP growth and inflation, we find that models with large numbers of predictors do not outperform models with 13–14 hand-picked predictors. Factor-augmented models and equal-weighted combinations of single-predictor mixed-data sampling regressions are a better choice for dealing with large numbers of predictors than Bayesian VARs.     

On the forecasting performance of a small-scale DSGE model

Dynamic stochastic general equilibrium (DSGE) models have recently become standard tools for policy analysis. Nevertheless, their forecasting properties have still barely been explored. In this article, we address this problem by examining the quality of forecasts of the key U.S. economic variables: the three-month Treasury bill yield, the GDP growth rate and GDP price index inflation, from a small-size DSGE model, trivariate vector autoregression (VAR) models and the Philadelphia Fed Survey of Professional Forecasters (SPF). The ex post forecast errors are evaluated on the basis of the data from the period 1994–2006. We apply the Philadelphia Fed “Real-Time Data Set for Macroeconomists” to ensure that the data used in estimating the DSGE and VAR models was comparable to the information available to the SPF.Overall, the results are mixed. When comparing the root mean squared errors for some forecast horizons, it appears that the DSGE model outperforms the other methods in forecasting the GDP growth rate. However, this characteristic turned out to be statistically insignificant. Most of the SPF's forecasts of GDP price index inflation and the short-term interest rate are better than those from the DSGE and VAR models.     

New Methods for Forecasting Inflation,Applied to the US*

Models for the 12‐month‐ahead US rate of inflation, measured by the chain‐weighted consumer expenditure deflator, are estimated for 1974–98 and subsequent pseudo out‐of‐sample forecasting performance is examined. Alternative forecasting approaches for different information sets are compared with benchmark univariate autoregressive models, and substantial out‐performance is demonstrated including against Stock and Watson's unobserved components‐stochastic volatility model. Three key ingredients to the out‐performance are: including equilibrium correction component terms in relative prices; introducing nonlinearities to proxy state‐dependence in the inflation process and replacing the information criterion, commonly used in VARs to select lag length, with a ‘parsimonious longer lags’ parameterization. Forecast pooling or averaging also improves forecast performance.     

The Incremental Predictive Information Associated with Using Theoretical New Keynesian DSGE Models vs. Simple Linear Econometric Models*

In this paper we construct output gap and inflation predictions using a variety of dynamic stochastic general equilibrium (DSGE) sticky price models. Predictive density accuracy tests related to the test discussed in Corradi and Swanson [Journal of Econometrics (2005a), forthcoming] as well as predictive accuracy tests due to Diebold and Mariano [Journal of Business and Economic Statistics (1995) , Vol. 13, pp. 253–263]; and West [Econometrica (1996) , Vol. 64, pp. 1067–1084] are used to compare the alternative models. A number of simple time‐series prediction models (such as autoregressive and vector autoregressive (VAR) models) are additionally used as strawman models. Given that DSGE model restrictions are routinely nested within VAR models, the addition of our strawman models allows us to indirectly assess the usefulness of imposing theoretical restrictions implied by DSGE models on unrestricted econometric models. With respect to predictive density evaluation, our results suggest that the standard sticky price model discussed in Calvo [Journal of Monetary Economics (1983), Vol. XII, pp. 383–398] is not outperformed by the same model augmented either with information or indexation, when used to predict the output gap. On the other hand, there are clear gains to using the more recent models when predicting inflation. Results based on mean square forecast error analysis are less clear‐cut, although the standard sticky price model fares best at our longest forecast horizon of 3 years, it performs relatively poorly at shorter horizons. When the strawman time‐series models are added to the picture, we find that the DSGE models still fare very well, often outperforming our forecast competitions, suggesting that theoretical macroeconomic restrictions yield useful additional information for forming macroeconomic forecasts.     

Forecasting the UK economy with a medium-scale Bayesian VAR

We estimate a Bayesian VAR (BVAR) for the UK economy and assess its performance in forecasting GDP growth and CPI inflation in real time relative to forecasts from COMPASS, the Bank of England’s DSGE model, and other benchmarks. We find that the BVAR outperformed COMPASS when forecasting both GDP and its expenditure components. In contrast, their performances when forecasting CPI were similar. We also find that the BVAR density forecasts outperformed those of COMPASS, despite under-predicting inflation at most forecast horizons. Both models over-predicted GDP growth at all forecast horizons, but the issue was less pronounced in the BVAR. The BVAR’s point and density forecast performances are also comparable to those of a Bank of England in-house statistical suite for both GDP and CPI inflation, as well as to the official Inflation Report projections. Our results are broadly consistent with the findings of similar studies for other advanced economies.     

Information criteria for impulse response function matching estimation of DSGE models

We propose new information criteria for impulse response function matching estimators (IRFMEs). These estimators yield sampling distributions of the structural parameters of dynamic stochastic general equilibrium (DSGE) models by minimizing the distance between sample and theoretical impulse responses. First, we propose an information criterion to select only the responses that produce consistent estimates of the true but unknown structural parameters: the Valid Impulse Response Selection Criterion (VIRSC). The criterion is especially useful for mis-specified models. Second, we propose a criterion to select the impulse responses that are most informative about DSGE model parameters: the Relevant Impulse Response Selection Criterion (RIRSC). These criteria can be used in combination to select the subset of valid impulse response functions with minimal dimension that yields asymptotically efficient estimators. The criteria are general enough to apply to impulse responses estimated by VARs, local projections, and simulation methods. We show that the use of our criteria significantly affects estimates and inference about key parameters of two well-known new Keynesian DSGE models. Monte Carlo evidence indicates that the criteria yield gains in terms of finite sample bias as well as offering tests statistics whose behavior is better approximated by the first order asymptotic theory. Thus, our criteria improve existing methods used to implement IRFMEs.     

Large time-varying parameter VARs

In this paper, we develop methods for estimation and forecasting in large time-varying parameter vector autoregressive models (TVP-VARs). To overcome computational constraints, we draw on ideas from the dynamic model averaging literature which achieve reductions in the computational burden through the use forgetting factors. We then extend the TVP-VAR so that its dimension can change over time. For instance, we can have a large TVP-VAR as the forecasting model at some points in time, but a smaller TVP-VAR at others. A final extension lies in the development of a new method for estimating, in a time-varying manner, the parameter(s) of the shrinkage priors commonly-used with large VARs. These extensions are operationalized through the use of forgetting factor methods and are, thus, computationally simple. An empirical application involving forecasting inflation, real output and interest rates demonstrates the feasibility and usefulness of our approach.     

Measuring output gap nowcast uncertainty

We propose a methodology for gauging the uncertainty in output gap nowcasts across a large number of commonly-deployed vector autoregressive (VAR) specifications for inflation and the output gap. Our approach utilises many output gap measures to construct ensemble nowcasts for inflation using a linear opinion pool. The predictive densities for the latent output gap utilise weights based on the ability of each specification to provide accurate probabilistic forecasts of inflation. In an application based on US real-time data, nowcasting over the out-of-sample evaluation period from 1991q2 to 2010q1, we demonstrate that a system of bivariate VARs produces well-calibrated ensemble densities for inflation, in contrast to univariate autoregressive benchmarks. The implied nowcast densities for the output gap are multimodal and indicate a considerable degree of uncertainty. For example, we assess the probability of a negative output gap at around 45% between 2004 and 2007. Despite the Greenspan policy regime, there still remained a substantial risk that the nowcast for output was below potential in real time. We extend our methodology to include distinct output gap measures, based on alternative filters, and show that, in our application, the nowcast density for the output gap is sensitive to the detrending method.     

Forecasting large datasets with Bayesian reduced rank multivariate models

The paper addresses the issue of forecasting a large set of variables using multivariate models. In particular, we propose three alternative reduced rank forecasting models and compare their predictive performance for US time series with the most promising existing alternatives, namely, factor models, large‐scale Bayesian VARs, and multivariate boosting. Specifically, we focus on classical reduced rank regression, a two‐step procedure that applies, in turn, shrinkage and reduced rank restrictions, and the reduced rank Bayesian VAR of Geweke ( 1996 ). We find that using shrinkage and rank reduction in combination rather than separately improves substantially the accuracy of forecasts, both when the whole set of variables is to be forecast and for key variables such as industrial production growth, inflation, and the federal funds rate. The robustness of this finding is confirmed by a Monte Carlo experiment based on bootstrapped data. We also provide a consistency result for the reduced rank regression valid when the dimension of the system tends to infinity, which opens the way to using large‐scale reduced rank models for empirical analysis. Copyright © 2010 John Wiley & Sons, Ltd.     

Empirically-transformed linear opinion pools

The linear opinion pool (LOP) produces potentially non-Gaussian combination forecast densities. In this paper, we propose a computationally convenient transformation for the LOP to mirror the non-Gaussianity exhibited by the target variable. Our methodology involves a Smirnov transform to reshape the LOP combination forecasts using the empirical cumulative distribution function. We illustrate our empirically transformed opinion pool (EtLOP) approach with an application examining quarterly real-time forecasts for U.S. inflation evaluated on a sample from 1990:1 to 2020:2. EtLOP improves performance by approximately 10% to 30% in terms of the continuous ranked probability score across forecasting horizons.     

Optimal forecasts in the presence of structural breaks

This paper considers the problem of forecasting under continuous and discrete structural breaks and proposes weighting observations to obtain optimal forecasts in the MSFE sense. We derive optimal weights for one step ahead forecasts. Under continuous breaks, our approach largely recovers exponential smoothing weights. Under discrete breaks, we provide analytical expressions for optimal weights in models with a single regressor, and asymptotically valid weights for models with more than one regressor. It is shown that in these cases the optimal weight is the same across observations within a given regime and differs only across regimes. In practice, where information on structural breaks is uncertain, a forecasting procedure based on robust optimal weights is proposed. The relative performance of our proposed approach is investigated using Monte Carlo experiments and an empirical application to forecasting real GDP using the yield curve across nine industrial economies.     

Combining survey long-run forecasts and nowcasts with BVAR forecasts using relative entropy

This paper constructs hybrid forecasts that combine forecasts from vector autoregressive (VAR) model(s) with both short- and long-term expectations from surveys. Specifically, we use the relative entropy to tilt one-step-ahead and long-horizon VAR forecasts to match the nowcasts and long-horizon forecasts from the Survey of Professional Forecasters. We consider a variety of VAR models, ranging from simple fixed-parameter to time-varying parameters. The results across models indicate meaningful gains in multi-horizon forecast accuracy relative to model forecasts that do not incorporate long-term survey conditions. Accuracy improvements are achieved for a range of variables, including those that are not tilted directly but are affected through spillover effects from tilted variables. The accuracy gains for hybrid inflation forecasts from simple VARs are substantial, statistically significant, and competitive to time-varying VARs, univariate benchmarks, and survey forecasts. We view our proposal as an indirect approach to accommodating structural change and moving end points.     

The Contribution of Structural Break Models to Forecasting Macroeconomic Series

This paper compares the forecasting performance of models that have been proposed for forecasting in the presence of structural breaks. They differ in their treatment of the break process, the model applied in each regime and the out‐of‐sample probability of a break. In an extensive empirical evaluation, we demonstrate the presence of breaks and their importance for forecasting. We find no single model that consistently works best in the presence of breaks. In many cases, the formal modeling of the break process is important in achieving a good forecast performance. However, there are also many cases where rolling window forecasts perform well. Copyright © 2014 John Wiley & Sons, Ltd.     

Forecast on silver futures linked with structural breaks and day-of-the-week effect

Silver future is crucial to global financial markets. However, the existing literature rarely considers the impacts of structural breaks and day-of-the-week effect simultaneously on the volatility of silver future price. Based on heterogeneous autoregressive (HAR) theory, we establish six new type heterogeneous autoregressive (HAR) models by incorporating structural breaks and day-of-the-week effect to forecast the volatility. The empirical results indicate that new models’ accuracy is better than the original HAR model. We find that structural breaks and the day-of-the-week effect contain much forecasting information on silver forecasting. In addition, structural breaks have a positive effect on the silver futures’ volatility. Day-of-the-week effect has a significantly negative influence on silver futures’ price volatility, especially in the mid-term and the long-term. Our works is the first to combine the structural breaks and day-of-the-week effect to identify more market information. This paper provides a better forecasting method to predict silver future volatility.     

Forecasting with Medium and Large Bayesian VARS

This paper is motivated by the recent interest in the use of Bayesian VARs for forecasting, even in cases where the number of dependent variables is large. In such cases factor methods have been traditionally used, but recent work using a particular prior suggests that Bayesian VAR methods can forecast better. In this paper, we consider a range of alternative priors which have been used with small VARs, discuss the issues which arise when they are used with medium and large VARs and examine their forecast performance using a US macroeconomic dataset containing 168 variables. We find that Bayesian VARs do tend to forecast better than factor methods and provide an extensive comparison of the strengths and weaknesses of various approaches. Typically, we find that the simple Minnesota prior forecasts well in medium and large VARs, which makes this prior attractive relative to computationally more demanding alternatives. Our empirical results show the importance of using forecast metrics based on the entire predictive density, instead of relying solely on those based on point forecasts. Copyright © 2011 John Wiley & Sons, Ltd.     

Efficient estimation of Bayesian VARMAs with time‐varying coefficients

Empirical work in macroeconometrics has been mostly restricted to using vector autoregressions (VARs), even though there are strong theoretical reasons to consider general vector autoregressive moving averages (VARMAs). A number of articles in the last two decades have conjectured that this is because estimation of VARMAs is perceived to be challenging and proposed various ways to simplify it. Nevertheless, VARMAs continue to be largely dominated by VARs, particularly in terms of developing useful extensions. We address these computational challenges with a Bayesian approach. Specifically, we develop a Gibbs sampler for the basic VARMA, and demonstrate how it can be extended to models with time‐varying vector moving average (VMA) coefficients and stochastic volatility. We illustrate the methodology through a macroeconomic forecasting exercise. We show that in a class of models with stochastic volatility, VARMAs produce better density forecasts than VARs, particularly for short forecast horizons.     

Evaluating Point and Density Forecasts of DSGE Models

This paper investigates the accuracy of forecasts from four dynamic stochastic general equilibrium (DSGE) models for inflation, output growth and the federal funds rate using a real‐time dataset synchronized with the Fed's Greenbook projections. Conditioning the model forecasts on the Greenbook nowcasts leads to forecasts that are as accurate as the Greenbook projections for output growth and the federal funds rate. Only for inflation are the model forecasts dominated by the Greenbook projections. A comparison with forecasts from Bayesian vector autoregressions shows that the economic structure of the DSGE models which is useful for the interpretation of forecasts does not lower the accuracy of forecasts. Combining forecasts of several DSGE models increases precision in comparison to individual model forecasts. Comparing density forecasts with the actual distribution of observations shows that DSGE models overestimate uncertainty around point forecasts. Copyright © 2013 John Wiley & Sons, Ltd.     

Steady‐state modeling and macroeconomic forecasting quality

Vector autoregressions (VARs) with informative steady‐state priors are standard forecasting tools in empirical macroeconomics. This study proposes (i) an adaptive hierarchical normal‐gamma prior on steady states, (ii) a time‐varying steady‐state specification which accounts for structural breaks in the unconditional mean, and (iii) a generalization of steady‐state VARs with fat‐tailed and heteroskedastic error terms. Empirical analysis, based on a real‐time dataset of 14 macroeconomic variables, shows that, overall, the hierarchical steady‐state specifications materially improve out‐of‐sample forecasting for forecasting horizons longer than 1 year, while the time‐varying specifications generate superior forecasts for variables with significant changes in their unconditional mean.     

Forecasting crude oil prices with DSGE models

In this study, we conducted an oil prices forecasting competition among a set of structural models, including vector autoregression and dynamic stochastic general equilibrium (DSGE) models. Our results highlight two principles. First, forecasts should exploit the fact that real oil prices are mean reverting over long horizons. Second, models should not replicate the high volatility of the oil prices observed in samples. By following these principles, we show that an oil sector DSGE model performs much better at real oil price forecasting than random walk or vector autoregression.     

Forecasting inflation using commodity price aggregates

This paper shows that for five small commodity-exporting countries that have adopted inflation targeting monetary policies, world commodity price aggregates have predictive power for their CPI and PPI inflation, particularly once possible structural breaks are taken into account. This conclusion is robust to using either disaggregated or aggregated commodity price indexes (although the former perform better), the currency denomination of the commodity prices, and to using mixed-frequency data. In pseudo out-of-sample forecasting, commodity indexes outperform the random walk and AR(1) processes, although the improvements over the latter are sometimes modest.