Evaluating FOMC forecasts

Monetary policy outcomes have improved since the early 1980s. One factor contributing to the improvement is that Federal Reserve policymakers began reporting economic forecasts to Congress in 1979. These forecasts indicate what the Federal Open Market Committee (FOMC) members think will be the likely consequence of their policies. We evaluate the accuracy of the FOMC forecasts relative to private sector forecasts, the forecasts of the Research Staff at the Board of Governors, and a naïve alternative. We find that the FOMC output forecasts were better than the naïve model and at least as good as those of the private sector and the Fed staff. The FOMC inflation forecasts were more accurate than the private sector forecasts and the naïve model; for the period ending in 1996, however, they were not as accurate as Fed staff inflation forecasts.     

EVALUATING MACROECONOMIC FORECASTS: A CONCISE REVIEW OF SOME RECENT DEVELOPMENTS

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.     

The Influence and Policy Signalling Role of FOMC Forecasts

Federal Open Market Committee (FOMC) policymakers have published macroeconomic forecasts since 1979 and we examine the effects of FOMC inflation forecasts using a structural VAR model. First, we assess whether they influence private inflation expectations. Second, we investigate the underlying mechanism at work and whether they convey policy signals. We provide original evidence that FOMC inflation forecasts influence private ones. We also find that the influencing effect of FOMC forecasts does not come through current Fed rate changes, that FOMC forecasts affect private expectations in a different way than current policy decisions, and that FOMC forecasts are informative about future Fed rate movements.     

GDP forecasts: Informational asymmetry of the SPF and FOMC minutes

This paper emphasizes asymmetric information about the U.S. economy between the FOMC and SPF. Following Stekler and Symington (2016), it extends their text-based FOMC minutes index (FMI) of economic outlook to 1986-2016. Following Ericsson (2016), it employs truncation adjustment indicators and reinterprets the FMI calibrations as the policy-makers forecasts of the GDP growth, which carry information about the staff Greenbook forecasts prepared prior to the bi-quarterly FOMC meetings. Tests confirm unbiasedness and rationality of these forecasts. The encompassing tests indicate that both the FMI and SPF forecasts contain unique information beyond their alternative’s information set and can be weighted equally. The orthogonality tests suggest that the SPF efficiently use all their information set but could gain if the FOMC minutes were published without a lag, while the policy-makers rely mostly on their projections made earlier in the meetings, and could benefit from incorporating the already published SPF forecasts.     

Evaluating a three-dimensional panel of point forecasts: The Bank of England Survey of External Forecasters

This article provides a first analysis of the forecasts of inflation and GDP growth obtained from the Bank of England's Survey of External Forecasters, considering both the survey average forecasts published in the quarterly Inflation Report, and the individual survey responses, recently made available by the Bank. These comprise a conventional incomplete panel dataset, with an additional dimension arising from the collection of forecasts at several horizons; both point forecasts and density forecasts are collected. The inflation forecasts show good performance in tests of unbiasedness and efficiency, albeit over a relatively calm period for the UK economy, and there is considerable individual heterogeneity. For GDP growth, inaccurate real-time data and their subsequent revisions are seen to cause serious difficulties for forecast construction and evaluation, although the forecasts are again unbiased. There is evidence that some forecasters have asymmetric loss functions.     

Gauging the uncertainty of the economic outlook using historical forecasting errors: The Federal Reserve’s approach

The Federal Open Market Committee (FOMC) of the U.S. Federal Reserve regularly publishes participants’ qualitative assessments of forecast uncertainty, expressed relative to that seen on average in the past. The benchmarks used for these historical comparisons are the average root mean squared forecast errors (RMSEs) made by various private and government forecasters over the past twenty years. This paper documents how these benchmarks are constructed and discusses some of their properties. We draw several conclusions. First, if past performance is a reasonable guide to future accuracy, considerable uncertainty surrounds macroeconomic projections. Second, different forecasters have similar accuracy. Third, estimates of uncertainty about future real activity and interest rates are now considerably greater than prior to the financial crisis; in contrast, estimates of inflation accuracy have changed little. Finally, fan charts, constructed under certain assumptions and viewed in conjunction with the FOMC’s qualitative assessments, provide a reasonable approximation to future uncertainty.     

Forecasting the UK economy with a medium-scale Bayesian VAR

We estimate a Bayesian VAR (BVAR) for the UK economy and assess its performance in forecasting GDP growth and CPI inflation in real time relative to forecasts from COMPASS, the Bank of England’s DSGE model, and other benchmarks. We find that the BVAR outperformed COMPASS when forecasting both GDP and its expenditure components. In contrast, their performances when forecasting CPI were similar. We also find that the BVAR density forecasts outperformed those of COMPASS, despite under-predicting inflation at most forecast horizons. Both models over-predicted GDP growth at all forecast horizons, but the issue was less pronounced in the BVAR. The BVAR’s point and density forecast performances are also comparable to those of a Bank of England in-house statistical suite for both GDP and CPI inflation, as well as to the official Inflation Report projections. Our results are broadly consistent with the findings of similar studies for other advanced economies.     

Introducing the Bank of Canada staff economic projections database

We present a new, publicly available database of real-time data and forecasts from the Bank of Canada's staff economic projections, which will be updated on an annual basis. We describe the data construct, its variables, coverage, and frequency. We then provide a forecast evaluation for gross domestic product (GDP) growth, consumer price index (CPI) inflation and the policy rate since 1982: We compare the staff's forecasts with those from commonly used time series models estimated with the real-time data, and with forecasts from other professional forecasters, and provide standard bias tests. Finally, we study changes in predictability of the Canadian economy following the announcement of the inflation-targeting regime in 1991. Our data set is unprecedented outside the USA, and our evidence is particularly interesting, as it covers over 30 years of staff forecasts, two severe recessions, and different monetary policy regimes.     

How reliable are forecasts of GDP growth and inflation for countries with limited coverage?

This paper analyses the performance of GDP growth and inflation forecasts for 25 transition countries between 1994 and 2007, as provided by 13 international institutions, including multilateral, private and academic forecasters. The empirical results show that there is a positive correlation between the number of forecasters covering a given country and the forecast accuracy. Simple combined forecasts are shown to be unbiased and more accurate than most of the individual forecasters, although also inefficient. However, only a few institutions provide efficient and unbiased forecasts, with just one out of 13 forecasters providing both unbiased and efficient forecasts of both GDP growth and inflation in the observed period. The directional analysis shows a correct forecast of the change in the forecast indicator in over two thirds of cases. However, the eventual outcome is within the range of available forecasts in less than half of the cases, with more than 40% of outcomes for GDP growth above the highest forecast. Encouragingly, forecasts are shown to be improving over time and becoming more accurate with the increase in the number of forecasting institutions – forecast accuracy measured by mean absolute error improves by 0.3 percentage points for growth and by 0.2 percentage points for inflation for each additional institution providing forecasts.     

An evaluation of the forecasts of the federal reserve: a pooled approach

The Federal Reserve Greenbook forecasts of real GDP, inflation and unemployment are analysed for the period 1974–1997. We consider whether these forecasts exhibit systematic bias, and whether efficient use is made of information, that is, whether revisions to these forecasts over time are predictable. Rather than analyse the forecasts separately for each horizon of interest, we discuss and implement methods that pool information over horizons. We conclude that there is evidence of systematic bias and of forecast smoothing of the inflation forecasts. Copyright © 2007 John Wiley & Sons, Ltd.     

Inflation forecasts and core inflation measures: Where is the information on future inflation?

This paper uses the forecast from a random walk model of inflation as a benchmark to test and compare the forecast performance of several alternatives of future inflation, including the Greenbook forecast by the Fed staff, the Survey of Professional Forecasters median forecast, CPI inflation minus food and energy, CPI weighted median inflation, and CPI trimmed mean inflation. The Greenbook forecast was found in previous literature to be a better forecast than other private sector forecasts. Our results indicate that both the Greenbook and the Survey of Professional Forecasters median forecasts of inflation and core inflation measures may contain better information than forecasts from a random walk model. The Greenbook's superiority appears to have declined against other forecasts and core inflation measures.     

Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts

If ‘learning by doing’ is important for macro-forecasting, newcomers might be different from regular, established participants. Stayers may also differ from the soon-to-leave. We test these conjectures for macro-forecasters’ point predictions of output growth and inflation, and for their histogram forecasts. Histogram forecasts of inflation by both joiners and leavers are found to be less accurate, especially if we suppose that joiners take time to learn. For GDP growth, there is no evidence of differences between the groups in terms of histogram forecast accuracy, although GDP point forecasts by leavers are less accurate. These findings are predicated on forecasters being homogeneous within groups. Allowing for individual fixed effects suggests fewer differences, including leavers’ inflation histogram forecasts being no less accurate.     

On the forecasting performance of a small-scale DSGE model

Dynamic stochastic general equilibrium (DSGE) models have recently become standard tools for policy analysis. Nevertheless, their forecasting properties have still barely been explored. In this article, we address this problem by examining the quality of forecasts of the key U.S. economic variables: the three-month Treasury bill yield, the GDP growth rate and GDP price index inflation, from a small-size DSGE model, trivariate vector autoregression (VAR) models and the Philadelphia Fed Survey of Professional Forecasters (SPF). The ex post forecast errors are evaluated on the basis of the data from the period 1994–2006. We apply the Philadelphia Fed “Real-Time Data Set for Macroeconomists” to ensure that the data used in estimating the DSGE and VAR models was comparable to the information available to the SPF.Overall, the results are mixed. When comparing the root mean squared errors for some forecast horizons, it appears that the DSGE model outperforms the other methods in forecasting the GDP growth rate. However, this characteristic turned out to be statistically insignificant. Most of the SPF's forecasts of GDP price index inflation and the short-term interest rate are better than those from the DSGE and VAR models.     

Inflation forecast contracts

We introduce a new type of incentive contract for central bankers: inflation forecast contracts, which make central bankers׳ remunerations contingent on the precision of their inflation forecasts. We show that such contracts enable central bankers to influence inflation expectations more effectively, thus facilitating more successful stabilization of current inflation. Inflation forecast contracts improve the accuracy of inflation forecasts, but have adverse consequences for output. On balance, paying central bankers according to their forecasting performance improves welfare. Optimal inflation forecast contracts stipulate high rewards for accurate forecasts.     

Forecasting macroeconomic risks

We construct risks around consensus forecasts of real GDP growth, unemployment, and inflation. We find that risks are time-varying, asymmetric, and partly predictable. Tight financial conditions forecast downside growth risk, upside unemployment risk, and increased uncertainty around the inflation forecast. Growth vulnerability arises as the conditional mean and conditional variance of GDP growth are negatively correlated: downside risks are driven by lower mean and higher variance when financial conditions tighten. Similarly, employment vulnerability arises as the conditional mean and conditional variance of unemployment are positively correlated, with tighter financial conditions corresponding to higher forecasted unemployment and higher variance around the consensus forecast.     

How accurate are government forecasts of economic fundamentals? The case of Taiwan

A government’s ability to forecast key economic fundamentals accurately can affect business confidence, consumer sentiment, and foreign direct investment, among others. A government forecast based on an econometric model is replicable, whereas one that is not fully based on an econometric model is non-replicable. Governments typically provide non-replicable forecasts (or expert forecasts) of economic fundamentals, such as the inflation rate and real GDP growth rate.In this paper, we develop a methodology for evaluating non-replicable forecasts. We argue that in order to do so, one needs to retrieve from the non-replicable forecast its replicable component, and that it is the difference in accuracy between these two that matters. An empirical example to forecast economic fundamentals for Taiwan shows the relevance of the proposed methodological approach. Our main finding is that the undocumented knowledge of the Taiwanese government reduces forecast errors substantially.     

Content horizons for univariate time-series forecasts

This paper investigates the maximum horizon at which conditioning information can be shown to have value for univariate time series forecasts. In particular, we consider the problem of determining the horizon beyond which forecasts from univariate time series models of stationary processes add nothing to the forecast implicit in the unconditional mean. We refer to this as the content horizon for forecasts, and provide a formal definition of the corresponding forecast content function at horizons s=1,… S. This function depends upon parameter estimation uncertainty as well as on autocorrelation structure of the process. We show that for autoregressive processes it is possible to give an asymptotic expression for the forecast content function, and show by simulation that the expression gives a good approximation even at modest sample sizes. The results are applied to the growth rate of GDP and to inflation, using US and Canadian data.     

Why big fiscal deficits and low inflation can coexist

• ▀ The surge in government debt caused by ballooning fiscal deficits is a necessary response to the coronavirus crisis. But we doubt this will lead to a burst of inflation in the advanced economies (AEs), let alone a debt crisis.
• ▀ Our fiscal forecasts assume AEs’ budget deficits averaged 20% of GDP or so in Q2. However, our deficit forecasts point to a sharp narrowing thereafter and for public debt as a share of GDP to peak in 2021.
• ▀ The risks around this forecast skew firmly towards deficits remaining wide, reflecting the balance of risks around our GDP forecasts and the possibility that governments allow some fiscal slippage.
• ▀ A slower narrowing of fiscal deficits than we forecast wouldn't automatically lead to a period of above-target inflation. Indeed, we wouldn't be surprised if larger-than-expected deficits were associated with weak inflation.
• ▀ High levels of corporate debt and weak labour markets raise the risk of private sector retrenchment ahead. In that case, large and sustained fiscal deficits may be needed to fill the vacuum and prevent GDP and inflation from falling. As has been the case in Japan over the past 25 years, large deficits over coming years could be associated with weak GDP growth and below-target inflation.
• ▀ If economies begin to overheat but governments keep fiscal policy loose, inflation could, of course, pick up. But central bank tightening would offset it. We believe the risk of sustained inflation overshoots is limited unless monetary policy were made subservient to governments’ own objectives. And we think the risk of central banks losing independence remains slim.

     

Fiscal Surprises at the FOMC

We examine a new set of U.S. fiscal forecasts from the FOMC briefing books. These forecasts are precisely those that were presented to monetary policymakers, and include frequently-updated estimates covering six complete business cycles and several fiscal-policy regimes. We detail the performances of forecast federal expenditures, revenues, surpluses, and structural surpluses in terms of their accuracy, bias, and efficiency. We find that forecast errors can be large economically, even at relatively short forecast horizons. While economic activity became less volatile after 1990, fiscal policy became harder to forecast. Finally, cyclically-adjusted deficit forecasts appear to be over-optimistic around both peaks and troughs of the business cycle, suggesting that fiscal policy is counter-cyclical in downturns and pro-cyclical in the early stages of recoveries.     

Euro area real-time density forecasting with financial or labor market frictions

We compare real-time density forecasts for the euro area using three DSGE models. The benchmark is the Smets and Wouters model, and its forecasts of real GDP growth and inflation are compared with those from two extensions. The first adds financial frictions and expands the observables to include a measure of the external finance premium. The second allows for the extensive labor-market margin and adds the unemployment rate to the observables. The main question that we address is whether these extensions improve the density forecasts of real GDP and inflation and their joint forecasts up to an eight-quarter horizon. We find that adding financial frictions leads to a deterioration in the forecasts, with the exception of longer-term inflation forecasts and the period around the Great Recession. The labor market extension improves the medium- to longer-term real GDP growth and shorter- to medium-term inflation forecasts weakly compared with the benchmark model.