Predicting relative forecasting performance: An empirical investigation

The relative performances of forecasting models change over time. This empirical observation raises two questions. First, is the relative performance itself predictable? Second, if so, can it be exploited in order to improve the forecast accuracy? We address these questions by evaluating the predictive abilities of a wide range of economic variables for two key US macroeconomic aggregates, namely industrial production and inflation, relative to simple benchmarks. We find that business cycle indicators, financial conditions, uncertainty and measures of past relative performances are generally useful for explaining the models’ relative forecasting performances. In addition, we conduct a pseudo-real-time forecasting exercise, where we use the information about the conditional performance for model selection and model averaging. The newly proposed strategies deliver sizable improvements over competitive benchmark models and commonly-used combination schemes. The gains are larger when model selection and averaging are based on both financial conditions and past performances measured at the forecast origin date.     

Assessing DSGE model nonlinearities

We develop a new class of time series models to identify nonlinearities in the data and to evaluate DSGE models. U.S. output growth and the federal funds rate display nonlinear conditional mean dynamics, while inflation and nominal wage growth feature conditional heteroskedasticity. We estimate a DSGE model with asymmetric wage and price adjustment costs and use predictive checks to assess its ability to account for these nonlinearities. While it is able to match the nonlinear inflation and wage dynamics, thanks to the estimated downward wage and price rigidities, these do not spill over to output growth or the interest rate.     

New perspectives on forecasting inflation in emerging market economies: An empirical assessment

We use a broad-range set of inflation models and pseudo out-of-sample forecasts to assess their predictive ability among 14 emerging market economies (EMEs) at different horizons (1–12 quarters ahead) with quarterly data over the period 1980Q1-2016Q4. We find, in general, that a simple arithmetic average of the current and three previous observations (the RW-AO model) consistently outperforms its standard competitors—based on the root mean squared prediction error (RMSPE) and on the accuracy in predicting the direction of change. These include conventional models based on domestic factors, existing open-economy Phillips curve-based specifications, factor-augmented models, and time-varying parameter models. Often, the RMSPE and directional accuracy gains of the RW-AO model are shown to be statistically significant. Our results are robust to forecast combinations, intercept corrections, alternative transformations of the target variable, different lag structures, and additional tests of (conditional) predictability. We argue that the RW-AO model is successful among EMEs because it is a straightforward method to downweight later data, which is a useful strategy when there are unknown structural breaks and model misspecification.     

On the real-time predictive content of financial condition indices for growth

We provide evidence on the real-time predictive content of the National Financial Conditions Index (NFCI), for conditional quantiles of U.S. real GDP growth. Our work is distinct from the literature in two specific ways. First, we construct (unofficial) real-time vintages of the NFCI. This allows us to conduct out-of-sample analysis without introducing the kind of look-ahead biases that are naturally introduced when using a single current vintage. We then develop methods for conducting asymptotic inference on tests of equal tick loss between nested quantile regression models when the data are subject to revision. We conclude by evaluating the real-time predictive content of NFCI vintages for quantiles of real GDP growth. While our results largely reinforce the literature, we find gains to using real-time vintages leading up to recessions—precisely when policymakers need such a monitoring device.     

Does judgment improve macroeconomic density forecasts?

This paper presents empirical evidence on how judgmental adjustments affect the accuracy of macroeconomic density forecasts. Judgment is defined as the difference between professional forecasters’ densities and the forecast densities from statistical models. Using entropic tilting, we evaluate whether judgments about the mean, variance and skew improve the accuracy of density forecasts for UK output growth and inflation. We find that not all judgmental adjustments help. Judgments about point forecasts tend to improve density forecast accuracy at short horizons and at times of heightened macroeconomic uncertainty. Judgments about the variance hinder at short horizons, but can improve tail risk forecasts at longer horizons. Judgments about skew in general take value away, with gains seen only for longer horizon output growth forecasts when statistical models took longer to learn that downside risks had reduced with the end of the Great Recession. Overall, density forecasts from statistical models prove hard to beat.     

A comprehensive evaluation of macroeconomic forecasting methods

We employ datasets for seven developed economies and consider four classes of multivariate forecasting models in order to extend and enhance the empirical evidence in the macroeconomic forecasting literature. The evaluation considers forecasting horizons of between one quarter and two years ahead. We find that the structural model, a medium-sized DSGE model, provides accurate long-horizon US and UK inflation forecasts. We strike a balance between being comprehensive and producing clear messages by applying meta-analysis regressions to 2,976 relative accuracy comparisons that vary with the forecasting horizon, country, model class and specification, number of predictors, and evaluation period. For point and density forecasting of GDP growth and inflation, we find that models with large numbers of predictors do not outperform models with 13–14 hand-picked predictors. Factor-augmented models and equal-weighted combinations of single-predictor mixed-data sampling regressions are a better choice for dealing with large numbers of predictors than Bayesian VARs.     

Are macroeconomic density forecasts informative?

We consider whether survey density forecasts (such as the inflation and output growth histograms of the US Survey of Professional Forecasters) are superior to unconditional density forecasts. The unconditional forecasts assume that the average level of uncertainty that has been experienced in the past will continue to prevail in the future, whereas the SPF projections ought to be adapted to the current conditions and the outlook at each forecast origin. The SPF forecasts might be expected to outperform the unconditional densities at the shortest horizons, but it transpires that such is not the case for the aggregate forecasts of either variable, or for the majority of the individual respondents for forecasting inflation.     

The effects of real and nominal uncertainty on inflation and output growth: some garch‐m evidence

In this paper we use GARCH‐M methods to test four hypotheses about the effects of real and nominal uncertainty on average inflation and output growth in the United States from 1948 to 1996. We find no evidence that higher inflation uncertainty or higher output growth uncertainty raises the average inflation rate. We also find no support for the idea that more risky output growth is associated with a higher average real growth rate. Our key result is that in a variety of models and sample periods, inflation uncertainty significantly lowers real output growth. Copyright © 2000 John Wiley & Sons, Ltd.     

Inflation Uncertainty, Output Growth Uncertainty and Macroeconomic Performance

We use a bivariate generalized autoregressive conditionally heteroskedastic (GARCH) model of inflation and output growth to examine the causality relationship among nominal uncertainty, real uncertainty and macroeconomic performance measured by the inflation and output growth rates. The application of the constant conditional correlation GARCH(1,1) model leads to a number of interesting conclusions. First, inflation does cause negative welfare effects, both directly and indirectly, i.e. via the inflation uncertainty channel. Secondly, in some countries, more inflation uncertainty provides an incentive to Central Banks to surprise the public by raising inflation unexpectedly. Thirdly, in contrast to the assumptions of some macroeconomic models, business cycle variability and the rate of economic growth are related. More variability in the business cycle leads to more output growth.     

Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts

If ‘learning by doing’ is important for macro-forecasting, newcomers might be different from regular, established participants. Stayers may also differ from the soon-to-leave. We test these conjectures for macro-forecasters’ point predictions of output growth and inflation, and for their histogram forecasts. Histogram forecasts of inflation by both joiners and leavers are found to be less accurate, especially if we suppose that joiners take time to learn. For GDP growth, there is no evidence of differences between the groups in terms of histogram forecast accuracy, although GDP point forecasts by leavers are less accurate. These findings are predicated on forecasters being homogeneous within groups. Allowing for individual fixed effects suggests fewer differences, including leavers’ inflation histogram forecasts being no less accurate.     

Measuring output gap nowcast uncertainty

We propose a methodology for gauging the uncertainty in output gap nowcasts across a large number of commonly-deployed vector autoregressive (VAR) specifications for inflation and the output gap. Our approach utilises many output gap measures to construct ensemble nowcasts for inflation using a linear opinion pool. The predictive densities for the latent output gap utilise weights based on the ability of each specification to provide accurate probabilistic forecasts of inflation. In an application based on US real-time data, nowcasting over the out-of-sample evaluation period from 1991q2 to 2010q1, we demonstrate that a system of bivariate VARs produces well-calibrated ensemble densities for inflation, in contrast to univariate autoregressive benchmarks. The implied nowcast densities for the output gap are multimodal and indicate a considerable degree of uncertainty. For example, we assess the probability of a negative output gap at around 45% between 2004 and 2007. Despite the Greenspan policy regime, there still remained a substantial risk that the nowcast for output was below potential in real time. We extend our methodology to include distinct output gap measures, based on alternative filters, and show that, in our application, the nowcast density for the output gap is sensitive to the detrending method.     

Density Forecasts With Midas Models

We propose a parametric block wild bootstrap approach to compute density forecasts for various types of mixed‐data sampling (MIDAS) regressions. First, Monte Carlo simulations show that predictive densities for the various MIDAS models derived from the block wild bootstrap approach are more accurate in terms of coverage rates than predictive densities derived from either a residual‐based bootstrap approach or by drawing errors from a normal distribution. This result holds whether the data‐generating errors are normally independently distributed, serially correlated, heteroskedastic or a mixture of normal distributions. Second, we evaluate density forecasts for quarterly US real output growth in an empirical exercise, exploiting information from typical monthly and weekly series. We show that the block wild bootstrapping approach, applied to the various MIDAS regressions, produces predictive densities for US real output growth that are well calibrated. Moreover, relative accuracy, measured in terms of the logarithmic score, improves for the various MIDAS specifications as more information becomes available. Copyright © 2016 John Wiley & Sons, Ltd.     

A predictability test for a small number of nested models

We introduce quasi-likelihood ratio tests for one sided multivariate hypotheses to evaluate the null that a parsimonious model performs equally well as a small number of models which nest the benchmark. The limiting distributions of the test statistics are non-standard. For critical values we consider: (i) bootstrapping and (ii) simulations assuming normality of the mean square prediction error difference. The proposed tests have good size and power properties compared with existing equal and superior predictive ability tests for multiple model comparison. We apply our tests to study the predictive ability of a Phillips curve type for the US core inflation.     

A time-varying volatility approach to modeling the phillips curve: A cross-country analysis

This research examines the Phillips curve price adjustment mechanism allowing for the conditional variance of inflation to be time varying. Specifically, we estimate ARCH and GARCH models of inflation for Canada, Japan, and the U.K. The results suggest that an increase in the conditional variability of inflation leads to higher levels of inflation. In addition, inclusion of inflation variability in the Phillips curve model results in a higher weight being attributed to the output gap than in traditional models. (JEF E24)     

The asymmetric effects of uncertainty on inflation and output growth

We study the effects of growth volatility and inflation volatility on average rates of output growth and inflation for post‐war US data. Our results suggest that increased growth uncertainty is associated with significantly lower average growth, while higher inflation uncertainty is significantly negatively correlated with lower output growth and lower average inflation. Both inflation and growth display evidence of significant asymmetric response to positive and negative shocks of equal magnitude. Copyright © 2004 John Wiley & Sons, Ltd.     

Forecasting macroeconomic time series: LASSO-based approaches and their forecast combinations with dynamic factor models

In a data-rich environment, forecasting economic variables amounts to extracting and organizing useful information from a large number of predictors. So far, the dynamic factor model and its variants have been the most successful models for such exercises. In this paper, we investigate a category of LASSO-based approaches and evaluate their predictive abilities for forecasting twenty important macroeconomic variables. These alternative models can handle hundreds of data series simultaneously, and extract useful information for forecasting. We also show, both analytically and empirically, that combing forecasts from LASSO-based models with those from dynamic factor models can reduce the mean square forecast error (MSFE) further. Our three main findings can be summarized as follows. First, for most of the variables under investigation, all of the LASSO-based models outperform dynamic factor models in the out-of-sample forecast evaluations. Second, by extracting information and formulating predictors at economically meaningful block levels, the new methods greatly enhance the interpretability of the models. Third, once forecasts from a LASSO-based approach are combined with those from a dynamic factor model by forecast combination techniques, the combined forecasts are significantly better than either dynamic factor model forecasts or the naïve random walk benchmark.     

Macroeconomic data transformations matter

In a low-dimensional linear regression setup, considering linear transformations/combinations of predictors does not alter predictions. However, when the forecasting technology either uses shrinkage or is nonlinear, it does. This is precisely the fabric of the machine learning (ML) macroeconomic forecasting environment. Pre-processing of the data translates to an alteration of the regularization – explicit or implicit – embedded in ML algorithms. We review old transformations and propose new ones, then empirically evaluate their merits in a substantial pseudo-out-sample exercise. It is found that traditional factors should almost always be included as predictors and moving average rotations of the data can provide important gains for various forecasting targets. Also, we note that while predicting directly the average growth rate is equivalent to averaging separate horizon forecasts when using OLS-based techniques, the latter can substantially improve on the former when regularization and/or nonparametric nonlinearities are involved.     

The Incremental Predictive Information Associated with Using Theoretical New Keynesian DSGE Models vs. Simple Linear Econometric Models*

In this paper we construct output gap and inflation predictions using a variety of dynamic stochastic general equilibrium (DSGE) sticky price models. Predictive density accuracy tests related to the test discussed in Corradi and Swanson [Journal of Econometrics (2005a), forthcoming] as well as predictive accuracy tests due to Diebold and Mariano [Journal of Business and Economic Statistics (1995) , Vol. 13, pp. 253–263]; and West [Econometrica (1996) , Vol. 64, pp. 1067–1084] are used to compare the alternative models. A number of simple time‐series prediction models (such as autoregressive and vector autoregressive (VAR) models) are additionally used as strawman models. Given that DSGE model restrictions are routinely nested within VAR models, the addition of our strawman models allows us to indirectly assess the usefulness of imposing theoretical restrictions implied by DSGE models on unrestricted econometric models. With respect to predictive density evaluation, our results suggest that the standard sticky price model discussed in Calvo [Journal of Monetary Economics (1983), Vol. XII, pp. 383–398] is not outperformed by the same model augmented either with information or indexation, when used to predict the output gap. On the other hand, there are clear gains to using the more recent models when predicting inflation. Results based on mean square forecast error analysis are less clear‐cut, although the standard sticky price model fares best at our longest forecast horizon of 3 years, it performs relatively poorly at shorter horizons. When the strawman time‐series models are added to the picture, we find that the DSGE models still fare very well, often outperforming our forecast competitions, suggesting that theoretical macroeconomic restrictions yield useful additional information for forming macroeconomic forecasts.     

Evaluating DSGE model forecasts of comovements

This paper develops and applies tools to assess multivariate aspects of Bayesian Dynamic Stochastic General Equilibrium (DSGE) model forecasts and their ability to predict comovements among key macroeconomic variables. We construct posterior predictive checks to evaluate conditional and unconditional density forecasts, in addition to checks for root-mean-squared errors and event probabilities associated with these forecasts. The checks are implemented on a three-equation DSGE model as well as the Smets and Wouters (2007) model using real-time data. We find that the additional features incorporated into the Smets–Wouters model do not lead to a uniform improvement in the quality of density forecasts and prediction of comovements of output, inflation, and interest rates.