Accuracy, unbiasedness and efficiency of professional macroeconomic forecasts: An empirical comparison for the G7

In this paper, we use survey data to analyze the accuracy, unbiasedness and efficiency of professional macroeconomic forecasts. We analyze a large panel of individual forecasts that has not previously been analyzed in the literature. We provide evidence on the properties of forecasts for all G7-countries and for four different macroeconomic variables. Our results show a high degree of dispersion of forecast accuracy across forecasters. We also find that there are large differences in the performances of forecasters, not only across countries but also across different macroeconomic variables. In general, the forecasts tend to be biased in situations where the forecasters have to learn about large structural shocks or gradual changes in the trend of a variable. Furthermore, while a sizable fraction of forecasters seem to smooth their GDP forecasts significantly, this does not apply to forecasts made for other macroeconomic variables.     

Evaluating the conditionality of judgmental forecasts

We propose a framework for evaluating the conditionality of forecasts. The crux of our framework is the observation that a forecast is conditional if revisions to the conditioning factor are incorporated faithfully into the remainder of the forecast. We consider whether the Greenbook, Blue Chip survey and Survey of Professional Forecasters exhibit systematic biases in the manner in which they incorporate interest rate projections into the forecasts of other macroeconomic variables. We do not find strong evidence of systematic biases in the three economic forecasts that we consider, as the interest rate projections in these forecasts appear to be incorporated efficiently into the forecasts of other economic variables.     

Analysts' Forecasts in Asian-Pacific Markets: The Relationship among Macroeconomic Factors, Accounting Systems, Bias and Accuracy

In this study, we describe determinants of accuracy/bias of analysts' forecasts in 13 economies of the Asian‐Pacific region. Examination of the accuracy of analysts' earnings forecasts allows us to judge how accounting systems and macroeconomic distinctions in this region affect earnings predictability. As many investors rely on analysts' earnings forecasts instead of producing their own, the growth of international investment means forecasts in non‐US markets will become increasingly important to investors worldwide. Using a sample of firms with data available on Global Vantage and I/B/E/S International, we find that the analysts on average have a pessimistic bias in Asian‐Pacific markets. We examine whether macroeconomic factors explain part of the difference in the size of analyst forecast errors, using the global competitiveness rankings of the World Economic Forum (WEF). We expect that those nations which are more open to foreign trade and investment and are ranked more highly by the WEF in its Global Competitiveness Index will also have more accurate analyst forecasts, as increased global competitiveness demands greater integration into the world economy, and such integration should lead to more transparent financial statements and more accurate earnings forecasts. Our findings are consistent with this prediction. We also find that countries with low book‐tax conformity have more accurate earnings forecasts.     

Nowcasting German GDP: Foreign factors,financial markets,and model averaging

This paper develops a nowcasting model for the German economy. The model outperforms a number of alternatives and produces forecasts not only for GDP but also for other key variables. We show that the inclusion of a foreign factor improves the model’s performance, while financial variables do not. Additionally, a comprehensive model averaging exercise reveals that factor extraction in a single model delivers slightly better results than averaging across models. Finally, we estimate a “news” index for the German economy in order to assess the overall performance of the model beyond forecast errors in GDP. The index is constructed as a weighted average of the nowcast errors related to each variable included in the model.     

Financial nowcasts and their usefulness in macroeconomic forecasting

Financial data often contain information that is helpful for macroeconomic forecasting, while multi-step forecast accuracy benefits from incorporating good nowcasts of macroeconomic variables. This paper considers the usefulness of financial nowcasts for making conditional forecasts of macroeconomic variables with quarterly Bayesian vector autoregressions (BVARs). When nowcasting quarterly financial variables’ values, we find that taking the average of the available daily data and a daily random walk forecast to complete the quarter typically outperforms other nowcasting approaches. Using real-time data, we find gains in out-of-sample forecast accuracy from the inclusion of financial nowcasts relative to unconditional forecasts, with further gains from the incorporation of nowcasts of macroeconomic variables. Conditional forecasts from quarterly BVARs augmented with financial nowcasts rival the forecast accuracy of mixed-frequency dynamic factor models and mixed-data sampling (MIDAS) models.     

Characteristics and implications of Chinese macroeconomic data revisions

The research examining macroeconomic data for developed economies suggests that an understanding of the nature of data revisions is important both for the production of accurate macroeconomic forecasts and for forecast evaluation. This paper focuses on Chinese data, for which there has been substantial debate about data quality for some time. The key finding in this paper is that, while it is true that the Chinese macroeconomic data revisions are not well-behaved, they are not very different from similarly-timed U.S. macroeconomic data revisions. The positive bias in Chinese real GDP revisions is a result of the fast-growing service sector, which is notably hard to measure in real time. A better understanding of the revisions process is particularly helpful for studies of the forecast errors from surveys of forecasters, where the choice of the vintage for outcomes may have an impact on the estimated forecast errors.     

The accuracy of IMF crises nowcasts

The International Monetary Fund (IMF) provides loans to countries in economic crises as a lender of last resort. IMF loan approvals are tied to policy reforms and quantitative targets that reflect the IMF’s crisis assessment. An extensive literature scrutinizes the efficacy of IMF loan programs, instead, we examine the accuracy of the IMF’s crisis assessments (nowcasts) that predicate program designs. Analyzing an unprecedented 602 IMF loan programs from 1992 to 2019, we contradict previous findings that IMF nowcasts are generally optimistic. Disentangling the structure of the IMF’s nowcast bias, we find the IMF systematically overestimates high-growth recoveries GDPs, while low-growth recoveries for low-income countries (LICs) are underestimated. In contrast, non-LICs’ nowcasts exhibit no statistically significant optimistic and pessimistic bias. Interestingly, shorter nowcast horizons do not improve accuracy, and GDP growth nowcasts improved substantially since 2013, while inflation nowcasts remain inefficient. We also isolate the sources of IMF nowcast inefficiencies according to ((i) program objectives, ((ii) program conditionality type, ((iii) geographic regions, ((iv) global crises, and ((v) geopolitics (elections, conflicts, and disasters).     

Examining the credibility of macroeconomic forecasts: Null of cointegration approach

In this study we examine the accuracy of forecasts of a select group of major macroeconomic variables, representing both the real and the financial sector of the economy. The theoretical foundations are similar to the one used to study exchange rate expectations, i.e. a verification of consistency and rationality in forecast formation. The empirical measure of accuracy is consistency in the expectation formation process, a precursor to rational forecasts. Here we examine the cointegration properties of the actual and forecast series (at multiple horizons) using the modern null of cointegration approach. A very reliable and continuos data set, the ASA-NBER survey is used. We find evidence of short (long) term expectational consistency (inconsistency) i.e. bandwagon effects and a mean reversion tendency in case of real variables, while the forecasts of financial variables are inconsistent across all forecast horizons.     

How accurate are government forecasts of economic fundamentals? The case of Taiwan

A government’s ability to forecast key economic fundamentals accurately can affect business confidence, consumer sentiment, and foreign direct investment, among others. A government forecast based on an econometric model is replicable, whereas one that is not fully based on an econometric model is non-replicable. Governments typically provide non-replicable forecasts (or expert forecasts) of economic fundamentals, such as the inflation rate and real GDP growth rate.In this paper, we develop a methodology for evaluating non-replicable forecasts. We argue that in order to do so, one needs to retrieve from the non-replicable forecast its replicable component, and that it is the difference in accuracy between these two that matters. An empirical example to forecast economic fundamentals for Taiwan shows the relevance of the proposed methodological approach. Our main finding is that the undocumented knowledge of the Taiwanese government reduces forecast errors substantially.     

How reliable are forecasts of GDP growth and inflation for countries with limited coverage?

This paper analyses the performance of GDP growth and inflation forecasts for 25 transition countries between 1994 and 2007, as provided by 13 international institutions, including multilateral, private and academic forecasters. The empirical results show that there is a positive correlation between the number of forecasters covering a given country and the forecast accuracy. Simple combined forecasts are shown to be unbiased and more accurate than most of the individual forecasters, although also inefficient. However, only a few institutions provide efficient and unbiased forecasts, with just one out of 13 forecasters providing both unbiased and efficient forecasts of both GDP growth and inflation in the observed period. The directional analysis shows a correct forecast of the change in the forecast indicator in over two thirds of cases. However, the eventual outcome is within the range of available forecasts in less than half of the cases, with more than 40% of outcomes for GDP growth above the highest forecast. Encouragingly, forecasts are shown to be improving over time and becoming more accurate with the increase in the number of forecasting institutions – forecast accuracy measured by mean absolute error improves by 0.3 percentage points for growth and by 0.2 percentage points for inflation for each additional institution providing forecasts.     

Forecasting with Bayesian Vector Autoregressions Estimated Using Professional Forecasts

We propose a Bayesian shrinkage approach for vector autoregressions (VARs) that uses short‐term survey forecasts as an additional source of information about model parameters. In particular, we augment the vector of dependent variables by their survey nowcasts, and claim that each variable modelled in the VAR and its nowcast are likely to depend in a similar way on the lagged dependent variables. In an application to macroeconomic data, we find that the forecasts obtained from a VAR fitted by our new shrinkage approach typically yield smaller mean squared forecast errors than the forecasts obtained from a range of benchmark methods. Copyright © 2015 John Wiley & Sons, Ltd.     

Macroeconomic forecasting for Australia using a large number of predictors

A popular approach to forecasting macroeconomic variables is to utilize a large number of predictors. Several regularization and shrinkage methods can be used to exploit such high-dimensional datasets, and have been shown to improve forecast accuracy for the US economy. To assess whether similar results hold for economies with different characteristics, an Australian dataset containing observations on 151 aggregate and disaggregate economic series as well as 185 international variables, is introduced. An extensive empirical study is carried out investigating forecasts at different horizons, using a variety of methods and with information sets containing an increasing number of predictors. In contrast to other countries the results show that it is difficult to forecast Australian key macroeconomic variables more accurately than some simple benchmarks. In line with other studies we also find that there is little to no improvement in forecast accuracy when the number of predictors is expanded beyond 20–40 variables and international factors do not seem to help.     

On the forecasting performance of a small-scale DSGE model

Dynamic stochastic general equilibrium (DSGE) models have recently become standard tools for policy analysis. Nevertheless, their forecasting properties have still barely been explored. In this article, we address this problem by examining the quality of forecasts of the key U.S. economic variables: the three-month Treasury bill yield, the GDP growth rate and GDP price index inflation, from a small-size DSGE model, trivariate vector autoregression (VAR) models and the Philadelphia Fed Survey of Professional Forecasters (SPF). The ex post forecast errors are evaluated on the basis of the data from the period 1994–2006. We apply the Philadelphia Fed “Real-Time Data Set for Macroeconomists” to ensure that the data used in estimating the DSGE and VAR models was comparable to the information available to the SPF.Overall, the results are mixed. When comparing the root mean squared errors for some forecast horizons, it appears that the DSGE model outperforms the other methods in forecasting the GDP growth rate. However, this characteristic turned out to be statistically insignificant. Most of the SPF's forecasts of GDP price index inflation and the short-term interest rate are better than those from the DSGE and VAR models.     

Financial information and macroeconomic forecasts

We study the forecasting power of financial variables for macroeconomic variables in 62 countries between 1980 and 2013. We find that financial variables such as credit growth, stock prices, and house prices have considerable predictive power for macroeconomic variables at the one- to four-quarter horizons. A forecasting model that includes financial variables outperforms the World Economic Outlook (WEO) forecasts in up to 85% of our sample countries at the four-quarter horizon. We also find that cross-country panel models produce more accurate out-of-sample forecasts than individual country models.     

EVALUATING MACROECONOMIC FORECASTS: A CONCISE REVIEW OF SOME RECENT DEVELOPMENTS

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.     

Are GDP forecasts optimal? Evidence on European countries

We assess the accuracy of real GDP growth forecasts released by governments and international organizations for European countries in the years 1999–2017. We implement three testing procedures characterized by different assumptions on the forecasters’ loss functions. First, we test forecast rationality within the traditional approach based on a quadratic loss function (Mincer and Zarnowitz, 1969). Second, following Elliott, Timmermann and Komunjer (2005), we test rationality by allowing for a flexible loss function where the shape parameter driving the extent of asymmetry is unknown and estimated from the empirical distribution of forecast errors. Lastly, we implement the tests proposed by Patton and Timmermann (2007a) that hold regardless of the functional form of the loss function. We conclude that governmental forecasts are biased and not rational under a symmetric and quadratic loss function, but they are optimal under more general assumptions on the loss function. We also find that the preferences of forecasters change with the forecasting horizon: when moving from one- to two-year-ahead forecasts, the optimistic bias increases and the parameter of asymmetry in the loss function significantly increases.     

Do IMF forecasts respect Okun’s law? Evidence for advanced and developing economies

This paper provides an assessment of the IMF’s unemployment forecasts, which have not received much scrutiny to date. The focus is on the internal consistency of the IMF’s growth and unemployment forecasts, and specifically on seeing whether the relationship between the two is consistent with the relationship in the data, i.e., with Okun’s Law. We find that the average performance is good, in the sense that the relationship between growth and unemployment forecasts is fairly comparable to that which prevails in the data: on average, the Okun coefficient in the forecasts mirrors the Okun coefficient in the data. Nevertheless, there is room for improvement, particularly in the year-ahead forecasts and for the group of middle-income countries. We show that a linear combination of Okun-based unemployment forecasts and WEO unemployment forecasts can deliver significant gains in forecast accuracy for developing economies.     

Nowcasting GDP with a pool of factor models and a fast estimation algorithm

We propose a novel mixed-frequency dynamic factor model with time-varying parameters and stochastic volatility for macroeconomic nowcasting and develop a fast estimation algorithm. This enables us to generate forecast densities based on a large space of factor models. We apply our framework to nowcast US GDP growth in real time. Our results reveal that stochastic volatility seems to improve the accuracy of point forecasts the most, compared to the constant-parameter factor model. These gains are most prominent during unstable periods such as the Covid-19 pandemic. Finally, we highlight indicators driving the US GDP growth forecasts and associated downside risks in real time.     

The Influence and Policy Signalling Role of FOMC Forecasts

Federal Open Market Committee (FOMC) policymakers have published macroeconomic forecasts since 1979 and we examine the effects of FOMC inflation forecasts using a structural VAR model. First, we assess whether they influence private inflation expectations. Second, we investigate the underlying mechanism at work and whether they convey policy signals. We provide original evidence that FOMC inflation forecasts influence private ones. We also find that the influencing effect of FOMC forecasts does not come through current Fed rate changes, that FOMC forecasts affect private expectations in a different way than current policy decisions, and that FOMC forecasts are informative about future Fed rate movements.     

Forecasting with Medium and Large Bayesian VARS

This paper is motivated by the recent interest in the use of Bayesian VARs for forecasting, even in cases where the number of dependent variables is large. In such cases factor methods have been traditionally used, but recent work using a particular prior suggests that Bayesian VAR methods can forecast better. In this paper, we consider a range of alternative priors which have been used with small VARs, discuss the issues which arise when they are used with medium and large VARs and examine their forecast performance using a US macroeconomic dataset containing 168 variables. We find that Bayesian VARs do tend to forecast better than factor methods and provide an extensive comparison of the strengths and weaknesses of various approaches. Typically, we find that the simple Minnesota prior forecasts well in medium and large VARs, which makes this prior attractive relative to computationally more demanding alternatives. Our empirical results show the importance of using forecast metrics based on the entire predictive density, instead of relying solely on those based on point forecasts. Copyright © 2011 John Wiley & Sons, Ltd.