Raiders of the lost high-frequency forecasts: New data and evidence on the efficiency of the Fed's forecasting

We introduce a new dataset of real gross domestic product (GDP) growth and core personal consumption expenditures (PCE) inflation forecasts produced by the staff of the Board of Governors of the Federal Reserve System. In contrast to the eight Greenbook forecasts a year the staff produce for Federal Open Market Committee (FOMC) meetings, our dataset has roughly weekly forecasts. We use these data to study whether the staff forecasts efficiently. Prespecified regressions of forecast errors on forecast revisions show the staff's GDP forecasts exhibit time-varying inefficiency between FOMC meetings, and also show some evidence for inefficient inflation forecasts.     

EVALUATING MACROECONOMIC FORECASTS: A CONCISE REVIEW OF SOME RECENT DEVELOPMENTS

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.     

Probabilistic population forecasting: Short to very long-term

Population forecasts are used by governments and the private sector for planning, with horizons up to about three generations (around 2100) for different purposes. The traditional methods are deterministic using scenarios, but probabilistic forecasts are desired to get an idea of accuracy, assess changes, and make decisions involving risks. In a significant breakthrough, since 2015, the United Nations has issued probabilistic population forecasts for all countries using a Bayesian methodology that we review here. Assessment of the social cost of carbon relies on long-term forecasts of carbon emissions, which in turn depend on even longer-range population and economic forecasts, to 2300. We extend the UN method to very-long range population forecasts by combining the statistical approach with expert review and elicitation. While the world population is projected to grow for the rest of this century, it will likely stabilize in the 22nd century and decline in the 23rd century.     

Forecasting Consumption: the Role of Consumer Confidence in Real Time with many Predictors

We study the role of consumer confidence in forecasting real personal consumption expenditure, and contribute to the extant literature in three substantive ways. First, we re‐examine existing empirical models of consumption and consumer confidence, not only at the quarterly frequency, but using monthly data as well. Second, we employ real‐time data in addition to commonly used revised vintages. Third, we investigate the role of consumer confidence in a rich information context. We produce forecasts of consumption expenditures with and without consumer confidence measures using a dynamic factor model and a large, real‐time, jagged‐edge dataset. In a robust way, we establish the important role of confidence surveys in improving the accuracy of consumption forecasts, manifesting primarily through the services component. During the recession of 2007–2009, sentiment is found to have a more pervasive effect on all components of aggregate consumption: durables, non‐durables and services. Copyright © 2015 John Wiley & Sons, Ltd.     

Forecasting inflation using time‐varying Bayesian model averaging

This paper presents a Bayesian model averaging regression framework for forecasting US inflation, in which the set of predictors included in the model is automatically selected from a large pool of potential predictors and the set of regressors is allowed to change over time. Using real‐time data on the 1960–2011 period, this model is applied to forecast personal consumption expenditures and gross domestic product deflator inflation. The results of this forecasting exercise show that, although it is not able to beat a simple random‐walk model in terms of point forecasts, it does produce superior density forecasts compared with a range of alternative forecasting models. Moreover, a sensitivity analysis shows that the forecasting results are relatively insensitive to prior choices and the forecasting performance is not affected by the inclusion of a very large set of potential predictors.     

Quantiles as optimal point forecasts

Loss functions play a central role in the theory and practice of forecasting. If the loss function is quadratic, the mean of the predictive distribution is the unique optimal point predictor. If the loss is symmetric piecewise linear, any median is an optimal point forecast. Quantiles arise as optimal point forecasts under a general class of economically relevant loss functions, which nests the asymmetric piecewise linear loss, and which we refer to as generalized piecewise linear (GPL). The level of the quantile depends on a generic asymmetry parameter which reflects the possibly distinct costs of underprediction and overprediction. Conversely, a loss function for which quantiles are optimal point forecasts is necessarily GPL. We review characterizations of this type in the work of Thomson, Saerens and Komunjer, and relate to proper scoring rules, incentive-compatible compensation schemes and quantile regression. In the empirical part of the paper, the relevance of decision theoretic guidance in the transition from a predictive distribution to a point forecast is illustrated using the Bank of England’s density forecasts of United Kingdom inflation rates, and probabilistic predictions of wind energy resources in the Pacific Northwest.     

Evaluating FOMC forecasts

Monetary policy outcomes have improved since the early 1980s. One factor contributing to the improvement is that Federal Reserve policymakers began reporting economic forecasts to Congress in 1979. These forecasts indicate what the Federal Open Market Committee (FOMC) members think will be the likely consequence of their policies. We evaluate the accuracy of the FOMC forecasts relative to private sector forecasts, the forecasts of the Research Staff at the Board of Governors, and a naïve alternative. We find that the FOMC output forecasts were better than the naïve model and at least as good as those of the private sector and the Fed staff. The FOMC inflation forecasts were more accurate than the private sector forecasts and the naïve model; for the period ending in 1996, however, they were not as accurate as Fed staff inflation forecasts.     

The challenges of pre-launch forecasting of adoption time series for new durable products

The successful introduction of new durable products plays an important part in helping companies to stay ahead of their competitors. Decisions relating to these products can be improved by the availability of reliable pre-launch forecasts of their adoption time series. However, producing such forecasts is a difficult, complex and challenging task, mainly because of the non-availability of past time series data relating to the product, and the multiple factors that can affect adoptions, such as customer heterogeneity, macroeconomic conditions following the product launch, and technological developments which may lead to the product’s premature obsolescence. This paper provides a critical review of the literature to examine what it can tell us about the relative effectiveness of three fundamental approaches to filling the data void : (i) management judgment, (ii) the analysis of judgments by potential customers, and (iii) formal models of the diffusion process. It then shows that the task of producing pre-launch time series forecasts of adoption levels involves a set of sub-tasks, which all involve either quantitative estimation or choice, and argues that the different natures of these tasks mean that the forecasts are unlikely to be accurate if a single method is employed. Nevertheless, formal models should be at the core of the forecasting process, rather than unstructured judgment. Gaps in the literature are identified, and the paper concludes by suggesting a research agenda so as to indicate where future research efforts might be employed most profitably.     

Seasonal exponential smoothing with damped trends : An application for production planning

This paper reviews a spreadsheet-based forecasting approach which a process industry manufacturer developed and implemented to link annual corporate forecasts with its manufacturing/distribution operations. First, we consider how this forecasting system supports overall production planning and why it must be compatible with corporate forecasts. We then review the results of substantial testing of variations on the Winters three-parameter exponential smoothing model on 28 actual product family time series. In particular, we evaluate whether the use of damping parameters improves forecast accuracy. The paper concludes that a Winters four-parameter model (i.e. the standard Winters three-parameter model augmented by a fourth parameter to damp the trend) provides the most accurate forecasts of the models evaluated. Our application confirms the fact that there are situations where the use of damped trend parameters in short-run exponential smoothing based forecasting models is beneficial.     

Evaluating Point and Density Forecasts of DSGE Models

This paper investigates the accuracy of forecasts from four dynamic stochastic general equilibrium (DSGE) models for inflation, output growth and the federal funds rate using a real‐time dataset synchronized with the Fed's Greenbook projections. Conditioning the model forecasts on the Greenbook nowcasts leads to forecasts that are as accurate as the Greenbook projections for output growth and the federal funds rate. Only for inflation are the model forecasts dominated by the Greenbook projections. A comparison with forecasts from Bayesian vector autoregressions shows that the economic structure of the DSGE models which is useful for the interpretation of forecasts does not lower the accuracy of forecasts. Combining forecasts of several DSGE models increases precision in comparison to individual model forecasts. Comparing density forecasts with the actual distribution of observations shows that DSGE models overestimate uncertainty around point forecasts. Copyright © 2013 John Wiley & Sons, Ltd.     

Assessing the uncertainty in central banks’ inflation outlooks

Recent research has found that macroeconomic survey forecasts of uncertainty exhibit several deficiencies, such as horizon-dependent biases and lower levels of accuracy than simple unconditional uncertainty forecasts. We examine the inflation uncertainty forecasts from the Bank of England, the Banco Central do Brasil, the Magyar Nemzeti Bank and the Sveriges Riksbank to assess whether central banks’ uncertainty forecasts might be subject to similar problems. We find that, while most central banks’ uncertainty forecasts also tend to be underconfident at short horizons and overconfident at longer horizons, they are mostly not significantly biased. Moreover, they tend to be at least as precise as unconditional uncertainty forecasts from two different approaches.     

Fiscal forecasting performance in an emerging economy: An empirical assessment of Brazil

This study makes use of Brazilian data to analyze government budget balance forecast errors. Besides the analysis of the quality and efficiency of budget balance forecasts, economic, political, and institutional and governance dimensions are explored. The findings show that the data forecasts have low quality and efficiency. Furthermore, it is observed that the budget forecast error is subject to a backward-looking effect, a bias in the economic growth forecasts, as well as cyclical fluctuations. Finally, electoral cycles represent a source of overestimated forecasts, and strong institutions and governance supported by the public are able to suppress opportunistic motivations in budget forecasts.     

Embrace the differences: Revisiting the PollyVote method of combining forecasts for U.S. presidential elections (2004 to 2020)

While combining forecasts is well-known to reduce error, the question of how to best combine forecasts remains. Prior research suggests that combining is most beneficial when relying on diverse forecasts that incorporate different information. Here, I provide evidence in support of this hypothesis by analyzing data from the PollyVote project, which has published combined forecasts of the popular vote in U.S. presidential elections since 2004. Prior to the 2020 election, the PollyVote revised its original method of combining forecasts by, first, restructuring individual forecasts based on their underlying information and, second, adding naïve forecasts as a new component method. On average across the last 100 days prior to the five elections from 2004 to 2020, the revised PollyVote reduced the error of the original specification by eight percent and, with a mean absolute error (MAE) of 0.8 percentage points, was more accurate than any of its component forecasts. The results suggest that, when deciding about which forecasts to include in the combination, forecasters should be more concerned about the component forecasts’ diversity than their historical accuracy.     

Financial forecasting and risky decisions: an experimental study grounded in Prospect theory

This study employed prospect theory to examine relationships between effort invested in developing financial forecasts and risk taking. Results of an experimental study indicated that the more effort subjects invested in developing forecasts, the more likely they were to use those forecasts as their reference points when evaluating venture performance. Results also indicated that subjects who used forecasts as their reference points and exerted greater effort developing those forecasts were more likely to take risky actions when performance fell below their reference points. This study is the first to link effort to the type of reference point used and the first to link effort and the use of financial forecasts to risky decisions. In addition, it is one of only a few studies to employ prospect theory to examine risk taking decisions subsequent to start-up. Its results enhance our understanding of risk taking, prospect theory and reference points.

     

Japan

As a consequence of his personal triumph in the July elections, Mr. Nakasone's term as Prime Minister has been extended beyond October when on existing rules he was due to step down. This suggests no immediate change in the economic policy of liberalising markets, control of public spending and fiscal restructuring rather than simple demand expansion. It also implies that, notwithstanding the vociferous opposition of export-oriented industry, the case for a strong yen, as a means of re-orienting demand towards domestic markets and curbing the trade surplus, is accepted by the government. Thus, despite continuing pressure from the US, Japan did not cut its discount rate over the summer. Nevertheless, with the economy flat and forecasts for 198617 being downgraded, a package of measures is scheduled for this autumn.     

Evaluating,Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation*

This paper proposes and analyses the Kullback–Leibler information criterion (KLIC) as a unified statistical tool to evaluate, compare and combine density forecasts. Use of the KLIC is particularly attractive, as well as operationally convenient, given its equivalence with the widely used Berkowitz likelihood ratio test for the evaluation of individual density forecasts that exploits the probability integral transforms. Parallels with the comparison and combination of point forecasts are made. This and related Monte Carlo experiments help draw out properties of combined density forecasts. We illustrate the uses of the KLIC in an application to two widely used published density forecasts for UK inflation, namely the Bank of England and NIESR ‘fan’ charts.     

Conditional Probabilistic Population Forecasting

Since policy-makers often prefer to think in terms of alternative scenarios, the question has arisen as to whether it is possible to make conditional population forecasts in a probabilistic context. This paper shows that it is both possible and useful to make these forecasts. We do this with two different kinds of examples. The first is the probabilistic analog of deterministic scenario analysis. Conditional probabilistic scenario analysis is essential for policy-makers because it allows them to answer "what if" type questions properly when outcomes are uncertain. The second is a new category that we call "future jump-off date forecasts". Future jump-off date forecasts are valuable because they show policy-makers the likelihood that crucial features of today's forecasts will also be present in forecasts made in the future.     

Complete subset regressions

This paper proposes a new method for combining forecasts based on complete subset regressions. For a given set of potential predictor variables we combine forecasts from all possible linear regression models that keep the number of predictors fixed. We explore how the choice of model complexity, as measured by the number of included predictor variables, can be used to trade off the bias and variance of the forecast errors, generating a setup akin to the efficient frontier known from modern portfolio theory. In an application to predictability of stock returns, we find that combinations of subset regressions can produce more accurate forecasts than conventional approaches based on equal-weighted forecasts (which fail to account for the dimensionality of the underlying models), combinations of univariate forecasts, or forecasts generated by methods such as bagging, ridge regression or Bayesian Model Averaging.     

Combining survey long-run forecasts and nowcasts with BVAR forecasts using relative entropy

This paper constructs hybrid forecasts that combine forecasts from vector autoregressive (VAR) model(s) with both short- and long-term expectations from surveys. Specifically, we use the relative entropy to tilt one-step-ahead and long-horizon VAR forecasts to match the nowcasts and long-horizon forecasts from the Survey of Professional Forecasters. We consider a variety of VAR models, ranging from simple fixed-parameter to time-varying parameters. The results across models indicate meaningful gains in multi-horizon forecast accuracy relative to model forecasts that do not incorporate long-term survey conditions. Accuracy improvements are achieved for a range of variables, including those that are not tilted directly but are affected through spillover effects from tilted variables. The accuracy gains for hybrid inflation forecasts from simple VARs are substantial, statistically significant, and competitive to time-varying VARs, univariate benchmarks, and survey forecasts. We view our proposal as an indirect approach to accommodating structural change and moving end points.     

Evaluating the conditionality of judgmental forecasts

We propose a framework for evaluating the conditionality of forecasts. The crux of our framework is the observation that a forecast is conditional if revisions to the conditioning factor are incorporated faithfully into the remainder of the forecast. We consider whether the Greenbook, Blue Chip survey and Survey of Professional Forecasters exhibit systematic biases in the manner in which they incorporate interest rate projections into the forecasts of other macroeconomic variables. We do not find strong evidence of systematic biases in the three economic forecasts that we consider, as the interest rate projections in these forecasts appear to be incorporated efficiently into the forecasts of other economic variables.