Are professional forecasters overconfident?

Do professional forecasters have an accurate sense of the uncertainties surrounding their own forecasts? This paper examines forecaster overconfidence by comparing ex ante, surveyed forecaster uncertainty with ex post, realised uncertainty based on the dispersion of an individual’s forecast errors. Unlike the literature that focuses on consensus forecasts, our focus is at the level of the individual forecaster. Using microdata from the three major surveys of professional forecasters (Euro Area, US and UK), we examine real GDP growth forecasts over the period 1999–2015. Our findings show that overconfidence dominates among individual forecasters, particularly for longer forecast horizons, and that individual forecasters appear to have little understanding of their own uncertainty.     

Efficient estimation of forecast uncertainty based on recent forecast errors

Multi-step-ahead forecasts of the forecast uncertainty of an individual forecaster are often based on the horizon-specific sample means of his recent squared forecast errors, where the number of past forecast errors available decreases one-to-one with the forecast horizon. In this paper, the efficiency gains from the joint estimation of forecast uncertainty for all horizons in such samples are investigated. If the forecast uncertainty is estimated by seemingly unrelated regressions, it turns out that the covariance matrix of the squared forecast errors does not have to be estimated, but simply needs to have a certain structure, which is a very useful property in small samples. Considering optimal and non-optimal forecasts, it is found that the efficiency gains can be substantial for longer horizons in small samples. The superior performance of the seemingly-unrelated-regressions approach is confirmed in several empirical applications.     

The Measurement and Behavior of Uncertainty: Evidence from the ECB Survey of Professional Forecasters

We examine matched point and density forecasts of output growth, inflation and unemployment from the ECB Survey of Professional Forecasters. We construct measures of uncertainty from individual histograms, and find that the measures display countercyclical behavior and have increased across all forecast horizons since 2007. We also derive measures of forecast dispersion and forecast accuracy, and find that they are not reliable proxies for uncertainty. There is, however, evidence of a meaningful co‐movement between uncertainty and aggregate point predictions for output growth and unemployment. These results are robust to changes in the composition of the survey respondents over time. Copyright © 2015 John Wiley & Sons, Ltd.     

Evaluating Point and Density Forecasts of DSGE Models

This paper investigates the accuracy of forecasts from four dynamic stochastic general equilibrium (DSGE) models for inflation, output growth and the federal funds rate using a real‐time dataset synchronized with the Fed's Greenbook projections. Conditioning the model forecasts on the Greenbook nowcasts leads to forecasts that are as accurate as the Greenbook projections for output growth and the federal funds rate. Only for inflation are the model forecasts dominated by the Greenbook projections. A comparison with forecasts from Bayesian vector autoregressions shows that the economic structure of the DSGE models which is useful for the interpretation of forecasts does not lower the accuracy of forecasts. Combining forecasts of several DSGE models increases precision in comparison to individual model forecasts. Comparing density forecasts with the actual distribution of observations shows that DSGE models overestimate uncertainty around point forecasts. Copyright © 2013 John Wiley & Sons, Ltd.     

Scoring rules and survey density forecasts

This article provides a practical evaluation of some leading density forecast scoring rules in the context of forecast surveys. We analyse the density forecasts of UK inflation obtained from the Bank of England’s Survey of External Forecasters, considering both the survey average forecasts published in the Bank’s quarterly Inflation Report, and the individual survey responses recently made available to researchers by the Bank. The density forecasts are collected in histogram format, and the ranked probability score (RPS) is shown to have clear advantages over other scoring rules. Missing observations are a feature of forecast surveys, and we introduce an adjustment to the RPS, based on the Yates decomposition, to improve its comparative measurement of forecaster performance in the face of differential non-response. The new measure, denoted RPS*, is recommended to analysts of forecast surveys.     

A look into the factor model black box: Publication lags and the role of hard and soft data in forecasting GDP

We derive forecast weights and uncertainty measures for assessing the roles of individual series in a dynamic factor model (DFM) for forecasting the euro area GDP from monthly indicators. The use of the Kalman smoother allows us to deal with publication lags when calculating the above measures. We find that surveys and financial data contain important information for the GDP forecasts beyond the monthly real activity measures. However, this is discovered only if their more timely publication is taken into account properly. Differences in publication lags play a very important role and should be considered in forecast evaluation.     

Research and Development,Uncertainty, and Analysts’ Forecasts: The Case of IAS 38

This study analyzes the consequences of the capitalization of development expenditures under IAS 38 on analysts’ earnings forecasts. We use unique hand‐collected data in a sample of highly research and development (R&D)‐intensive German‐listed firms over the period 2000–2007. We find that the capitalization of development costs is significantly associated with both higher individual analysts’ forecast errors and forecast dispersion. This suggests that the increasing complexity surrounding the capitalization of development costs negatively impacts forecast accuracy. However, for firms with high underlying environmental uncertainty, forecast errors are negatively associated with capitalized development expenditures. This indicates that the negative impact of increased complexity on forecast accuracy can be outweighed by the information contained in the signals from capitalized development costs when the underlying environmental uncertainty is high. The findings contribute to the ongoing controversial debate on the accounting for self‐generated intangible assets. Our results provide useful insights on the link between capitalization of development costs, environmental uncertainty, and analysts’ forecasts for accounting academics and practitioners alike.     

Does Forecast Combination Improve Norges Bank Inflation Forecasts?*

We develop a system that provides model‐based forecasts for inflation in Norway. We recursively evaluate quasi out‐of‐sample forecasts from a large suite of models from 1999 to 2009. The performance of the models are then used to derive quasi real time weights that are used to combine the forecasts. Our results indicate that a combination forecast improves upon the point forecasts from individual models. Furthermore, a combination forecast outperforms Norges Bank's own point forecast for inflation. The beneficial results are obtained using a trimmed weighted average. Some degree of trimming is required for the combination forecasts to outperform the judgmental forecasts from the policymaker.     

Rounding behaviour of professional macro-forecasters

The rounding of point forecasts of CPI inflation and the unemployment rate by U.S. Professional Forecasters is modest. There is little evidence that forecasts are rounded to a greater extent in response to higher perceived uncertainty surrounding future outcomes. There is clear evidence that the probability of decline forecasts are rounded: over half of the forecast probabilities of decline in the current quarter are multiples of ten. It is found here that the rounding of these probabilities correlates with worse accuracy, although it is also of note here that worse (less accurate) forecasters might round more as opposed to the degree of rounding per se worsening accuracy. By simulating the loss from rounding for a set of efficient forecasters, it is demonstrated that the explanation that respondents round otherwise efficient forecasts is implausible, and that the contribution of rounding is of minor importance.     

CONSTRUCTING OPTIMAL DENSITY FORECASTS FROM POINT FORECAST COMBINATIONS

Decision makers often observe point forecasts of the same variable computed, for instance, by commercial banks, IMF and the World Bank, but the econometric models used by such institutions are frequently unknown. This paper shows how to use the information available on point forecasts to compute optimal density forecasts. Our idea builds upon the combination of point forecasts under general loss functions and unknown forecast error distributions. We use real‐time data to forecast the density of US inflation. The results indicate that the proposed method materially improves the real‐time accuracy of density forecasts vis‐à‐vis those from the (unknown) individual econometric models. Copyright © 2013 John Wiley & Sons, Ltd.     

Do macroeconomic forecasters use macroeconomics to forecast?

Faced with an uncertain future, forecasters often rely on textbook relationships to build a coherent narrative for their macroeconomic forecasts. We focus on two cornerstones of modern macroeconomics – Okun’s law and the Phillips curve – and examine whether or not professionals forecast in a way that is consistent with these. Using microdata from the US, Euro Area, and UK surveys of professional forecasters, we examine forecasts over the period 1981-2017 at the level of the individual and across different time horizons. Our findings show that the majority of forecasters produce their forecasts in a manner that is consistent with macroeconomic theory.     

Does judgment improve macroeconomic density forecasts?

This paper presents empirical evidence on how judgmental adjustments affect the accuracy of macroeconomic density forecasts. Judgment is defined as the difference between professional forecasters’ densities and the forecast densities from statistical models. Using entropic tilting, we evaluate whether judgments about the mean, variance and skew improve the accuracy of density forecasts for UK output growth and inflation. We find that not all judgmental adjustments help. Judgments about point forecasts tend to improve density forecast accuracy at short horizons and at times of heightened macroeconomic uncertainty. Judgments about the variance hinder at short horizons, but can improve tail risk forecasts at longer horizons. Judgments about skew in general take value away, with gains seen only for longer horizon output growth forecasts when statistical models took longer to learn that downside risks had reduced with the end of the Great Recession. Overall, density forecasts from statistical models prove hard to beat.     

Forecasting (aggregate) demand for US commercial air travel

We analyze whether it is better to forecast air travel demand using aggregate data at (say) a national level, or to aggregate the forecasts derived for individual airports using airport-specific data. We compare the US Federal Aviation Administration’s (FAA) practice of predicting the total number of passengers using macroeconomic variables with an equivalently specified AIM (aggregating individual markets) approach. The AIM approach outperforms the aggregate forecasting approach in terms of its out-of-sample air travel demand predictions for different forecast horizons. Variants of AIM, where we restrict the coefficient estimates of some explanatory variables to be the same across individual airports, generally dominate both the aggregate and AIM approaches. The superior out-of-sample performances of these so-called quasi-AIM approaches depend on the trade-off between heterogeneity and estimation uncertainty. We argue that the quasi-AIM approaches exploit the heterogeneity across individual airports efficiently, without suffering from as much estimation uncertainty as the AIM approach.     

Does the Capitalization of Development Costs Improve Analyst Forecast Accuracy? Evidence from the UK

It has been documented that investments in Research and Development (R&D) are associated with increased errors and inaccuracy in earnings forecasts made by financial analysts. These deficiencies have been generally attributed to information complexity and the uncertainty of the future benefits of R&D. This paper examines whether the capitalization of development costs can reduce analyst uncertainty about the future economic outcome of R&D investments, provide outsiders with a better matching of future R&D‐related revenues and costs, and therefore promote accuracy in analyst forecasts. UK data is used, because accounting rules in the United Kingdom permitted firms to conditionally capitalize development costs even before the introduction of the International Financial Reporting Standards. The choice to expense R&D rather than conditionally capitalize development costs is found to relate positively to signed analyst forecast errors. This finding is robust to controlling for the influence of other factors that may affect errors, as well as for the influence of R&D investments on forecast errors. The decision to capitalize versus expense is not observed to have a significant influence on analyst forecast revisions. The findings are interpreted as evidence that the choice to capitalize as opposed to expense may help to reduce deficiencies in analyst forecasts; hence, is informative for users of financial statements. Increased informativeness is expected to have repercussions for the effectiveness with which analysts produce earnings forecasts, and, as a result, market efficiency.     

Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts

If ‘learning by doing’ is important for macro-forecasting, newcomers might be different from regular, established participants. Stayers may also differ from the soon-to-leave. We test these conjectures for macro-forecasters’ point predictions of output growth and inflation, and for their histogram forecasts. Histogram forecasts of inflation by both joiners and leavers are found to be less accurate, especially if we suppose that joiners take time to learn. For GDP growth, there is no evidence of differences between the groups in terms of histogram forecast accuracy, although GDP point forecasts by leavers are less accurate. These findings are predicated on forecasters being homogeneous within groups. Allowing for individual fixed effects suggests fewer differences, including leavers’ inflation histogram forecasts being no less accurate.     

Evaluating a three-dimensional panel of point forecasts: The Bank of England Survey of External Forecasters

This article provides a first analysis of the forecasts of inflation and GDP growth obtained from the Bank of England's Survey of External Forecasters, considering both the survey average forecasts published in the quarterly Inflation Report, and the individual survey responses, recently made available by the Bank. These comprise a conventional incomplete panel dataset, with an additional dimension arising from the collection of forecasts at several horizons; both point forecasts and density forecasts are collected. The inflation forecasts show good performance in tests of unbiasedness and efficiency, albeit over a relatively calm period for the UK economy, and there is considerable individual heterogeneity. For GDP growth, inaccurate real-time data and their subsequent revisions are seen to cause serious difficulties for forecast construction and evaluation, although the forecasts are again unbiased. There is evidence that some forecasters have asymmetric loss functions.     

Simple rules for combining forecasts: Some empirical results

This study is an empirical comparison of three rules for aggregating forecasts. The three combined forecasts evaluated are: a simple average forecast, a median forecast and a focus forecast. These combined forecasts are compared over four economic variables (housing starts, the index of industrial production, the unemployment rate and gross national product) using a set of previously published forecasts. The results indicate that an average forecast will not perform as well as previous studies indicate if all or most of the individual forecasts tend to over- or under-predict simultaneously. The median forecast also seems to be suspect in this case. There is little evidence to suggest that the median forecast is a viable alternative to the mean forecast. Focus forecasting, however, is found to perform well for all four variables. The evidence indicates that focus forecasting is a reasonable alternative to simple averaging.     

Modelling multi‐period inflation uncertainty using a panel of density forecasts

This paper examines the determinants of inflation forecast uncertainty using a panel of density forecasts from the Survey of Professional Forecasters (SPF). Based on a dynamic heterogeneous panel data model, we find that the persistence in forecast uncertainty is much less than what the aggregate time series data would suggest. In addition, the strong link between past forecast errors and current forecast uncertainty, as often noted in the ARCH literature, is largely lost in a multi‐period context with varying forecast horizons. We propose a novel way of estimating ‘news’ and its variance using the Kullback‐Leibler information, and show that the latter is an important determinant of forecast uncertainty. Our evidence suggests a strong relationship of forecast uncertainty with level of inflation, but not with forecaster discord or with the volatility of a number of other macroeconomic indicators. Copyright © 2006 John Wiley & Sons, Ltd.     

Are macroeconomic density forecasts informative?

We consider whether survey density forecasts (such as the inflation and output growth histograms of the US Survey of Professional Forecasters) are superior to unconditional density forecasts. The unconditional forecasts assume that the average level of uncertainty that has been experienced in the past will continue to prevail in the future, whereas the SPF projections ought to be adapted to the current conditions and the outlook at each forecast origin. The SPF forecasts might be expected to outperform the unconditional densities at the shortest horizons, but it transpires that such is not the case for the aggregate forecasts of either variable, or for the majority of the individual respondents for forecasting inflation.     

Population forecasting theory,methods and assessments of accuracy. The magnitude of error due to different vital processes in population forecasts

The accuracy of population forecasts depends in part upon the method chosen for forecasting the vital rates of fertility, mortality, and migration. Methods for handling the stochastic propagation of error calculations in demographic forecasting are hard to do precisely. This paper discusses this obstacle in stochastic cohort-component population forecasts. The uncertainty of forecasts is due to uncertain estimates of the jump-off population and to errors in the forecasts of the vital rates. Empirically based of each source are presented and propagated through a simplified analytical model of population growth that allows assessment of the role of each component in the total error. Numerical estimates based on the errors of an actual vector ARIMA forecast of the US female population. These results broadly agree with those of the analytical model. This work especially uncertainty in the fertility forecasts to be so much higher than that in the other sources that the latter can be ignored in the propagation of error calculations for those cohorts that are born after the jump-off year of the forecast. A methodology is therefore presented which far simplifies the propagation of error calculations. It is noted, however, that the uncertainty of the jump-off population, migration, and mortality in the propagation of error for those alive at the jump-off time of the forecast must still be considered.