On the forecasting performance of a small-scale DSGE model

Dynamic stochastic general equilibrium (DSGE) models have recently become standard tools for policy analysis. Nevertheless, their forecasting properties have still barely been explored. In this article, we address this problem by examining the quality of forecasts of the key U.S. economic variables: the three-month Treasury bill yield, the GDP growth rate and GDP price index inflation, from a small-size DSGE model, trivariate vector autoregression (VAR) models and the Philadelphia Fed Survey of Professional Forecasters (SPF). The ex post forecast errors are evaluated on the basis of the data from the period 1994–2006. We apply the Philadelphia Fed “Real-Time Data Set for Macroeconomists” to ensure that the data used in estimating the DSGE and VAR models was comparable to the information available to the SPF.Overall, the results are mixed. When comparing the root mean squared errors for some forecast horizons, it appears that the DSGE model outperforms the other methods in forecasting the GDP growth rate. However, this characteristic turned out to be statistically insignificant. Most of the SPF's forecasts of GDP price index inflation and the short-term interest rate are better than those from the DSGE and VAR models.     

Forecasting the UK economy: Alternative forecasting methodologies and the role of off-model information

How did DSGE model forecasts perform before, during and after the financial crisis, and what type of off-model information can improve the forecast accuracy? We tackle these questions by assessing the real-time forecast performance of a large DSGE model relative to statistical and judgmental benchmarks over the period from 2000 to 2013. The forecasting performances of all methods deteriorate substantially following the financial crisis. That is particularly evident for the DSGE model’s GDP forecasts, but augmenting the model with a measure of survey expectations made its GDP forecasts more accurate, which supports the idea that timely off-model information is particularly useful in times of financial distress.     

Open economy forecasting with a DSGE-VAR: Head to head with the RBNZ published forecasts

We construct a DSGE-VAR model for competing head to head with the long history of published forecasts of the Reserve Bank of New Zealand. We also construct a Bayesian VAR model with a Minnesota prior for forecast comparison. The DSGE-VAR model combines a structural DSGE model with a statistical VAR model based on the in-sample fit over the majority of New Zealand’s inflation-targeting period. We evaluate the real-time out-of-sample forecasting performance of the DSGE-VAR model, and show that the forecasts from the DSGE-VAR are competitive with the Reserve Bank of New Zealand’s published, judgmentally-adjusted forecasts. The Bayesian VAR model with a Minnesota prior also provides a competitive forecasting performance, and generally, with a few exceptions, out-performs both the DSGE-VAR and the Reserve Bank’s own forecasts.     

Marginalized Predictive Likelihood Comparisons of Linear Gaussian State‐Space Models with Applications to DSGE,DSGE‐VAR,and VAR Models

The predictive likelihood is useful for ranking models in forecast comparison exercises using Bayesian inference. We discuss how it can be estimated, by means of marzginalization, for any subset of the observables in linear Gaussian state‐space models. We compare macroeconomic density forecasts for the euro area of a DSGE model to those of a DSGE‐VAR, a BVAR and a multivariate random walk over 1999:Q1–2011:Q4. While the BVAR generally provides superior forecasts, its performance deteriorates substantially with the onset of the Great Recession. This is particularly notable for longer‐horizon real GDP forecasts, where the DSGE and DSGE‐VAR models perform better. Copyright © 2016 John Wiley & Sons, Ltd.     

Forecasting crude oil prices with DSGE models

In this study, we conducted an oil prices forecasting competition among a set of structural models, including vector autoregression and dynamic stochastic general equilibrium (DSGE) models. Our results highlight two principles. First, forecasts should exploit the fact that real oil prices are mean reverting over long horizons. Second, models should not replicate the high volatility of the oil prices observed in samples. By following these principles, we show that an oil sector DSGE model performs much better at real oil price forecasting than random walk or vector autoregression.     

DSGE forecasts of the lost recovery

The years following the Great Recession were challenging for forecasters. Unlike other deep downturns, this recession was not followed by a swift recovery, but instead generated a sizable and persistent output gap that was not accompanied by deflation as a traditional Phillips curve relationship would have predicted. Moreover, the zero lower bound and unconventional monetary policy generated an unprecedented policy environment. We document the actual real-time forecasting performance of the New York Fed dynamic stochastic general equilibrium (DSGE) model during this period and explain the results using the pseudo real-time forecasting performance results from a battery of DSGE models. We find the New York Fed DSGE model’s forecasting accuracy to be comparable to that of private forecasters, and notably better for output growth than the median forecasts from the FOMC’s Summary of Economic Projections. The model’s financial frictions were key in obtaining these results, as they implied a slow recovery following the financial crisis.     

Short-term forecasting of GDP with a DSGE model augmented by monthly indicators

DSGE models are useful tools for evaluating the impact of policy changes, but their use for (short-term) forecasting is still in its infancy. Besides theory-based restrictions, the timeliness of data is an important issue. Since DSGE models are based on quarterly data, they suffer from the publication lag of quarterly national accounts. In this paper we present a framework for the short-term forecasting of GDP based on a medium-scale DSGE model for a small open economy within a currency area. We utilize the information available in monthly indicators based on the approach proposed by Giannone et al. (2009). Using Austrian data, we find that the forecasting performance of the DSGE model can be improved considerably by incorporating monthly indicators, while still maintaining the story-telling capability of the model.     

Forecasting the UK economy with a medium-scale Bayesian VAR

We estimate a Bayesian VAR (BVAR) for the UK economy and assess its performance in forecasting GDP growth and CPI inflation in real time relative to forecasts from COMPASS, the Bank of England’s DSGE model, and other benchmarks. We find that the BVAR outperformed COMPASS when forecasting both GDP and its expenditure components. In contrast, their performances when forecasting CPI were similar. We also find that the BVAR density forecasts outperformed those of COMPASS, despite under-predicting inflation at most forecast horizons. Both models over-predicted GDP growth at all forecast horizons, but the issue was less pronounced in the BVAR. The BVAR’s point and density forecast performances are also comparable to those of a Bank of England in-house statistical suite for both GDP and CPI inflation, as well as to the official Inflation Report projections. Our results are broadly consistent with the findings of similar studies for other advanced economies.     

Tests of Predictive Ability for Vector Autoregressions Used for Conditional Forecasting

Many forecasts are conditional in nature. For example, a number of central banks routinely report forecasts conditional on particular paths of policy instruments. Even though conditional forecasting is common, there has been little work on methods for evaluating conditional forecasts. This paper provides analytical, Monte Carlo and empirical evidence on tests of predictive ability for conditional forecasts from estimated models. In the empirical analysis, we examine conditional forecasts obtained with a VAR in the variables included in the DSGE model of Smets and Wouters (American Economic Review 2007; 97 : 586–606). Throughout the analysis, we focus on tests of bias, efficiency and equal accuracy applied to conditional forecasts from VAR models. Copyright © 2016 John Wiley & Sons, Ltd.     

Fast and accurate yearly time series forecasting with forecast combinations

It has long been known that combination forecasting strategies produce superior out-of-sample forecasting performances. In the M4 forecasting competition, a very simple forecast combination strategy achieved third place on yearly time series. An analysis of the ensemble model and its component models suggests that the competitive accuracy comes from avoiding poor forecasts, rather than from beating the best individual models. Moreover, the simple ensemble model can be fitted very quickly, can easily scale horizontally with additional CPU cores or a cluster of computers, and can be implemented by users very quickly and easily. This approach might be of particular interest to users who need accurate yearly forecasts without being able to spend significant time, resources, or expertise on tuning models. Users of the R statistical programming language can access this modeling approach using the “forecastHybrid” package.     

Evaluating Point and Density Forecasts of DSGE Models

This paper investigates the accuracy of forecasts from four dynamic stochastic general equilibrium (DSGE) models for inflation, output growth and the federal funds rate using a real‐time dataset synchronized with the Fed's Greenbook projections. Conditioning the model forecasts on the Greenbook nowcasts leads to forecasts that are as accurate as the Greenbook projections for output growth and the federal funds rate. Only for inflation are the model forecasts dominated by the Greenbook projections. A comparison with forecasts from Bayesian vector autoregressions shows that the economic structure of the DSGE models which is useful for the interpretation of forecasts does not lower the accuracy of forecasts. Combining forecasts of several DSGE models increases precision in comparison to individual model forecasts. Comparing density forecasts with the actual distribution of observations shows that DSGE models overestimate uncertainty around point forecasts. Copyright © 2013 John Wiley & Sons, Ltd.     

Selecting structural innovations in DSGE models

Dynamic stochastic general equilibrium (DSGE) models are typically estimated assuming the existence of certain structural shocks that drive macroeconomic fluctuations. We analyze the consequences of estimating shocks that are “nonexistent” and propose a method to select the economic shocks driving macroeconomic uncertainty. Forcing these nonexisting shocks in estimation produces a downward bias in the estimated internal persistence of the model. We show how these distortions can be reduced by using priors for standard deviations whose support includes zero. The method allows us to accurately select shocks and estimate model parameters with high precision. We revisit the empirical evidence on an industry standard medium‐scale DSGE model and find that government and price markup shocks are innovations that do not generate statistically significant dynamics.     

Neural networks for GEFCom2017 probabilistic load forecasting

This report describes the forecasting model which was developed by team “4C” for the global energy forecasting competition 2017 (GEFCom2017), with some modifications added afterwards to improve its accuracy. The model is based on neural networks. Temperature scenarios obtained from historical data are used as inputs to the neural networks in order to create load scenarios, and these load scenarios are then transformed into quantiles. By using a feature selection approach that is based on a stepwise regression technique, a neural network based model is developed for each zone. Furthermore, a dynamic choice of the temperature scenarios is suggested. The feature selection and dynamic choice of the temperature scenarios can improve the quantile scores considerably, resulting in very accurate forecasts among the top teams.     

Analyzing fixed-event forecast revisions

It is common practice to evaluate fixed-event forecast revisions in macroeconomics by regressing current forecast revisions on one-period lagged forecast revisions. Under weak-form (forecast) efficiency, the correlation between the current and one-period lagged revisions should be zero. The empirical findings in the literature suggest that this null hypothesis of zero correlation is rejected frequently, and the correlation can be either positive (which is widely interpreted in the literature as “smoothing”) or negative (which is widely interpreted as “over-reacting”). We propose a methodology for interpreting such non-zero correlations in a straightforward and clear manner. Our approach is based on the assumption that numerical forecasts can be decomposed into both an econometric model and random expert intuition. We show that the interpretation of the sign of the correlation between the current and one-period lagged revisions depends on the process governing intuition, and the current and lagged correlations between intuition and news (or shocks to the numerical forecasts). It follows that the estimated non-zero correlation cannot be given a direct interpretation in terms of either smoothing or over-reaction.     

Forecasting volatility with time-varying leverage and volatility of volatility effects

Predicting volatility is of primary importance for business applications in risk management, asset allocation, and the pricing of derivative instruments. This paper proposes a measurement model that considers the possibly time-varying interaction of realized volatility and asset returns according to a bivariate model to capture its major characteristics: (i) the long-term memory of the volatility process, (ii) the heavy-tailedness of the distribution of returns, and (iii) the negative dependence of volatility and daily market returns. We assess the relevance of the effects of “the volatility of volatility” and time-varying “leverage” to the out-of-sample forecasting performance of the model, and evaluate the density of forecasts of market volatility. Empirical results show that our specification can outperform the benchmark HAR–GARCH model in terms of both point and density forecasts.     

Forecasts of US short-term interest rates: A flexible forecast combination approach

This paper develops a flexible approach to combine forecasts of future spot rates with forecasts from time-series models or macroeconomic variables. We find empirical evidence that, accounting for both regimes in interest rate dynamics, and combining forecasts from different models, helps improve the out-of-sample forecasting performance for US short-term rates. Imposing restrictions from the expectations hypothesis on the forecasting model are found to help at long forecasting horizons.     

The performance of alternative forecasting methods for SETAR models

We compare a number of methods that have been proposed in the literature for obtaining h-step ahead minimum mean square error forecasts for self-exciting threshold autoregressive (SETAR) models. These forecasts are compared to those from an AR model. The comparison of forecasting methods is made using Monte Carlo simulation. The Monte-Carlo method of calculating SETAR forecasts is generally at least as good as that of the other methods we consider. An exception is when the disturbances in the SETAR model come from a highly asymmetric distribution, when a Bootstrap method is to be preferred.An empirical application calculates multi-period forecasts from a SETAR model of US gross national product using a number of the forecasting methods. We find that whether there are improvements in forecast performance relative to a linear AR model depends on the historical epoch we select, and whether forecasts are evaluated conditional on the regime the process was in at the time the forecast was made.     

Forecasting macroeconomic time series: LASSO-based approaches and their forecast combinations with dynamic factor models

In a data-rich environment, forecasting economic variables amounts to extracting and organizing useful information from a large number of predictors. So far, the dynamic factor model and its variants have been the most successful models for such exercises. In this paper, we investigate a category of LASSO-based approaches and evaluate their predictive abilities for forecasting twenty important macroeconomic variables. These alternative models can handle hundreds of data series simultaneously, and extract useful information for forecasting. We also show, both analytically and empirically, that combing forecasts from LASSO-based models with those from dynamic factor models can reduce the mean square forecast error (MSFE) further. Our three main findings can be summarized as follows. First, for most of the variables under investigation, all of the LASSO-based models outperform dynamic factor models in the out-of-sample forecast evaluations. Second, by extracting information and formulating predictors at economically meaningful block levels, the new methods greatly enhance the interpretability of the models. Third, once forecasts from a LASSO-based approach are combined with those from a dynamic factor model by forecast combination techniques, the combined forecasts are significantly better than either dynamic factor model forecasts or the naïve random walk benchmark.     

Predicting economic contractions and expansions with the aid of professional forecasts

Traditional econometric models of economic contractions typically perform poorly in forecasting exercises. This criticism is also frequently levelled at professional forecast probabilities of contractions. This paper addresses the problem of incorporating the entire distribution of professional forecasts into an econometric model for forecasting contractions and expansions. A new augmented probit approach is proposed, involving the transformation of the distribution of professional forecasts into a ‘professional forecast’ prior for the economic data underlying the probit model. Since the object of interest is the relationship between the distribution of professional forecasts and the probit model’s economic-data dependent parameters, the solution avoids criticisms levelled at the accuracy of professional forecast based point estimates of contractions. An application to US real GDP data shows that the model yields significant forecast improvements relative to alternative approaches.     

An accurate and fully-automated ensemble model for weekly time series forecasting

Many businesses and industries require accurate forecasts for weekly time series nowadays. However, the forecasting literature does not currently provide easy-to-use, automatic, reproducible and accurate approaches dedicated to this task. We propose a forecasting method in this domain to fill this gap, leveraging state-of-the-art forecasting techniques, such as forecast combination, meta-learning, and global modelling. We consider different meta-learning architectures, algorithms, and base model pools. Based on all considered model variants, we propose to use a stacking approach with lasso regression which optimally combines the forecasts of four base models: a global Recurrent Neural Network (RNN) model, Theta, Trigonometric Box–Cox ARMA Trend Seasonal (TBATS), and Dynamic Harmonic Regression ARIMA (DHR-ARIMA), as it shows the overall best performance across seven experimental weekly datasets on four evaluation metrics. Our proposed method also consistently outperforms a set of benchmarks and state-of-the-art weekly forecasting models by a considerable margin with statistical significance. Our method can produce the most accurate forecasts, in terms of mean sMAPE, for the M4 weekly dataset among all benchmarks and all original competition participants.