Combining Forecasts from Nested Models*

Motivated by the common finding that linear autoregressive models often forecast better than models that incorporate additional information, this paper presents analytical, Monte Carlo and empirical evidence on the effectiveness of combining forecasts from nested models. In our analytics, the unrestricted model is true, but a subset of the coefficients is treated as being local‐to‐zero. This approach captures the practical reality that the predictive content of variables of interest is often low. We derive mean square error‐minimizing weights for combining the restricted and unrestricted forecasts. Monte Carlo and empirical analyses verify the practical effectiveness of our combination approach.     

Forecast Encompassing and Parameter Estimation*

A desirable property of a forecast is that it encompasses competing predictions, in the sense that the accuracy of the preferred forecast cannot be improved through linear combination with a rival prediction. In this paper, we investigate the impact of the uncertainty associated with estimating model parameters in‐sample on the encompassing properties of out‐of‐sample forecasts. Specifically, using examples of non‐nested econometric models, we show that forecasts from the true (but estimated) data generating process (DGP) do not encompass forecasts from competing mis‐specified models in general, particularly when the number of in‐sample observations is small. Following this result, we also examine the scope for achieving gains in accuracy by combining the forecasts from the DGP and mis‐specified models.     

Combining Density and Interval Forecasts: A Modest Proposal*

The finite mixture distribution is proposed as an appropriate statistical model for a combined density forecast. Its implications for measures of uncertainty and disagreement, and for combining interval forecasts, are described. Related proposals in the literature and applications to the U.S. Survey of Professional Forecasters are discussed.     

Evaluating,Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation*

This paper proposes and analyses the Kullback–Leibler information criterion (KLIC) as a unified statistical tool to evaluate, compare and combine density forecasts. Use of the KLIC is particularly attractive, as well as operationally convenient, given its equivalence with the widely used Berkowitz likelihood ratio test for the evaluation of individual density forecasts that exploits the probability integral transforms. Parallels with the comparison and combination of point forecasts are made. This and related Monte Carlo experiments help draw out properties of combined density forecasts. We illustrate the uses of the KLIC in an application to two widely used published density forecasts for UK inflation, namely the Bank of England and NIESR ‘fan’ charts.     

On the Construction of the European Economic Sentiment Indicator*

Economic sentiment surveys are carried out by all European Union member states and are often seen as early indicators for future economic developments. Based on these surveys, the European Commission constructs an aggregate European Economic Sentiment Indicator (ESI). This paper compares the ESI with more sophisticated aggregation schemes based on statistical methods: dynamic factor analysis and partial least squares. The indicator based on partial least squares clearly outperforms the other two indicators in terms of comovement with economic activity. In terms of forecast ability, the ESI, constructed in a rather ad hoc way, can compete with the other indicators.     

Levels,Differences and ECMs – Principles for Improved Econometric Forecasting*

Unit‐root testing can be a preliminary step in model development, an intermediate step, or an end in itself. Some researchers have questioned the value of any unit‐root and cointegration testing, arguing that restrictions based on theory are at least as effective. Such confusion is unsatisfactory. Needed is a set of principles that limit and define the role of the tacit knowledge of the model builders. In a forecasting context, we enumerate the various possible model selection strategies and, based on simulation and empirical evidence, recommend using these tests to improve the specification of an initial general vector autoregression model.     

A Simple Explanation of the Forecast Combination Puzzle*

This article presents a formal explanation of the forecast combination puzzle, that simple combinations of point forecasts are repeatedly found to outperform sophisticated weighted combinations in empirical applications. The explanation lies in the effect of finite‐sample error in estimating the combining weights. A small Monte Carlo study and a reappraisal of an empirical study by Stock and Watson [Federal Reserve Bank of Richmond Economic Quarterly (2003) Vol. 89/3, pp. 71–90] support this explanation. The Monte Carlo evidence, together with a large‐sample approximation to the variance of the combining weight, also supports the popular recommendation to ignore forecast error covariances in estimating the weight.     

An Introduction to Best Empirical Models when the Parameter Space is Infinite Dimensional*

Ploberger and Phillips (Econometrica, Vol. 71, pp. 627–673, 2003) proved a result that provides a bound on how close a fitted empirical model can get to the true model when the model is represented by a parameterized probability measure on a finite dimensional parameter space. The present note extends that result to cases where the parameter space is infinite dimensional. The results have implications for model choice in infinite dimensional problems and highlight some of the difficulties, including technical difficulties, presented by models of infinite dimension. Some implications for forecasting are considered and some applications are given, including the empirically relevant case of vector autoregression (VAR) models of infinite order.     

Bayesian Estimation of Willingness‐to‐pay Where Respondents Mis‐report Their Preferences*

We introduce a modified conditional logit model that takes account of uncertainty associated with mis‐reporting in revealed preference experiments estimating willingness‐to‐pay (WTP). Like Hausman et al. [Journal of Econometrics (1988) Vol. 87, pp. 239–269], our model captures the extent and direction of uncertainty by respondents. Using a Bayesian methodology, we apply our model to a choice modelling (CM) data set examining UK consumer preferences for non‐pesticide food. We compare the results of our model with the Hausman model. WTP estimates are produced for different groups of consumers and we find that modified estimates of WTP, that take account of mis‐reporting, are substantially revised downwards. We find a significant proportion of respondents mis‐reporting in favour of the non‐pesticide option. Finally, with this data set, Bayes factors suggest that our model is preferred to the Hausman model.     

Selecting a Nonlinear Time Series Model using Weighted Tests of Equal Forecast Accuracy*

Nonlinear time series models have become fashionable tools to describe and forecast a variety of economic time series. A closer look at reported empirical studies, however, reveals that these models apparently fit well in‐sample, but rarely show a substantial improvement in out‐of‐sample forecasts, at least over linear models. One of the many possible reasons for this finding is the use of inappropriate model selection criteria and forecast evaluation criteria. In this paper we therefore propose a novel criterion, which we believe does more justice to the very nature of nonlinear models. Simulations show that this criterion outperforms those criteria currently in use, in the sense that the true nonlinear model is more often found to perform better in out‐of‐sample forecasting than a benchmark linear model. An empirical illustration for US GDP emphasizes its relevance.     

Optimal forecasts in the presence of structural breaks

This paper considers the problem of forecasting under continuous and discrete structural breaks and proposes weighting observations to obtain optimal forecasts in the MSFE sense. We derive optimal weights for one step ahead forecasts. Under continuous breaks, our approach largely recovers exponential smoothing weights. Under discrete breaks, we provide analytical expressions for optimal weights in models with a single regressor, and asymptotically valid weights for models with more than one regressor. It is shown that in these cases the optimal weight is the same across observations within a given regime and differs only across regimes. In practice, where information on structural breaks is uncertain, a forecasting procedure based on robust optimal weights is proposed. The relative performance of our proposed approach is investigated using Monte Carlo experiments and an empirical application to forecasting real GDP using the yield curve across nine industrial economies.     

Forecasting a long memory process subject to structural breaks

We develop an easy-to-implement method for forecasting a stationary autoregressive fractionally integrated moving average (ARFIMA) process subject to structural breaks with unknown break dates. We show that an ARFIMA process subject to a mean shift and a change in the long memory parameter can be well approximated by an autoregressive (AR) model and suggest using an information criterion (AIC or Mallows’ _Cp$C_p$) to choose the order of the approximate AR model. Our method avoids the issue of estimation inaccuracy of the long memory parameter and the issue of spurious breaks in finite sample. Insights from our theoretical analysis are confirmed by Monte Carlo experiments, through which we also find that our method provides a substantial improvement over existing prediction methods. An empirical application to the realized volatility of three exchange rates illustrates the usefulness of our forecasting procedure. The empirical success of the HAR-RV model can be explained, from an econometric perspective, by our theoretical and simulation results.     

Selection of estimation window in the presence of breaks

In situations where a regression model is subject to one or more breaks it is shown that it can be optimal to use pre-break data to estimate the parameters of the model used to compute out-of-sample forecasts. The issue of how best to exploit the trade-off that might exist between bias and forecast error variance is explored and illustrated for the multivariate regression model under the assumption of strictly exogenous regressors. In practice when this assumption cannot be maintained and both the time and size of the breaks are unknown, the optimal choice of the observation window will be subject to further uncertainties that make exploiting the bias–variance trade-off difficult. To that end we propose a new set of cross-validation methods for selection of a single estimation window and weighting or pooling methods for combination of forecasts based on estimation windows of different lengths. Monte Carlo simulations are used to show when these procedures work well compared with methods that ignore the presence of breaks.     

Convergence of Prices and Rates of Inflation*

We consider how unit‐root and stationarity tests can be used to study the convergence of prices and rates of inflation. We show how the joint use of these tests in levels and first differences allows the researcher to distinguish between series that are converging and series that have already converged, and we set out a strategy to establish whether convergence occurs in relative prices or just in rates of inflation. Special attention is paid to the issue of whether a mean should be extracted in carrying out tests in first differences and whether there is an advantage to adopting a (Dickey–Fuller) unit‐root test based on deviations from the last observation. The asymptotic distribution of this last test statistic is given and Monte Carlo simulation experiments show that the test yields considerable power gains for highly persistent autoregressive processes with ‘relatively large’ initial conditions. The tests are applied to the monthly series of the consumer price index in the Italian regional capitals over the period 1970–2003.     

Nonlinear Correlograms and Partial Autocorrelograms*

This paper proposes neural network‐based measures of predictability in conditional mean, and then uses them to construct nonlinear analogues to autocorrelograms and partial autocorrelograms. In contrast to other measures of nonlinear dependence that rely on nonparametric estimation of densities or multivariate integration, our autocorrelograms are simple to calculate and appear to work well in relatively small samples.     

New Methods for Forecasting Inflation,Applied to the US*

Models for the 12‐month‐ahead US rate of inflation, measured by the chain‐weighted consumer expenditure deflator, are estimated for 1974–98 and subsequent pseudo out‐of‐sample forecasting performance is examined. Alternative forecasting approaches for different information sets are compared with benchmark univariate autoregressive models, and substantial out‐performance is demonstrated including against Stock and Watson's unobserved components‐stochastic volatility model. Three key ingredients to the out‐performance are: including equilibrium correction component terms in relative prices; introducing nonlinearities to proxy state‐dependence in the inflation process and replacing the information criterion, commonly used in VARs to select lag length, with a ‘parsimonious longer lags’ parameterization. Forecast pooling or averaging also improves forecast performance.     

Prewhitening Bias in HAC Estimation*

Heteroskedasticity and autocorrelation consistent (HAC) estimation commonly involves the use of prewhitening filters based on simple autoregressive models. In such applications, small sample bias in the estimation of autoregressive coefficients is transmitted to the recolouring filter, leading to HAC variance estimates that can be badly biased. The present paper provides an analysis of these issues using asymptotic expansions and simulations. The approach we recommend involves the use of recursive demeaning procedures that mitigate the effects of small‐sample autoregressive bias. Moreover, a commonly used restriction rule on the prewhitening estimates (that first‐order autoregressive coefficient estimates, or largest eigenvalues, >0.97 be replaced by 0.97) adversely interferes with the power of unit‐root and [ Kwiatkowski, Phillips, Schmidt and Shin (1992) Journal of Econometrics, Vol. 54, pp. 159–178] (KPSS) tests. We provide a new boundary condition rule that improves the size and power properties of these tests. Some illustrations of the effects of these adjustments on the size and power of KPSS testing are given. Using prewhitened HAC estimates and the new boundary condition rule, the KPSS test is consistent, in contrast to KPSS testing that uses conventional prewhitened HAC estimates [ Lee, J. S. (1996) Economics Letters, Vol. 51, pp. 131–137].     

Threshold Effects in Cointegrating Relationships*

In this paper, we introduce threshold‐type nonlinearities within a single‐equation cointegrating regression model and propose a testing procedure for testing the null hypothesis of linear cointegration vs. cointegration with threshold effects. Our framework allows the modelling of long‐run equilibrium relationships that may change according to the magnitude of a threshold variable assumed to be stationary and ergodic, and thus constitutes an attempt to deal econometrically with the potential presence of multiple equilibria. The framework is flexible enough to accommodate regressor endogeneity and serial correlation.     

Modeling data revisions: Measurement error and dynamics of “true” values

Policy makers must base their decisions on preliminary and partially revised data of varying reliability. Realistic modeling of data revisions is required to guide decision makers in their assessment of current and future conditions. This paper provides a new framework with which to model data revisions.Recent empirical work suggests that measurement errors typically have much more complex dynamics than existing models of data revisions allow. This paper describes a state-space model that allows for richer dynamics in these measurement errors, including the noise, news and spillover effects documented in this literature. We also show how to relax the common assumption that “true” values are observed after a few revisions.The result is a unified and flexible framework that allows for more realistic data revision properties, and allows the use of standard methods for optimal real-time estimation of trends and cycles. We illustrate the application of this framework with real-time data on US real output growth.     

A control function approach for testing the usefulness of trending variables in forecast models and linear regression

Many predictors employed in forecasting macroeconomic and finance variables display a great deal of persistence. Tests for determining the usefulness of these predictors are typically oversized, overstating their importance. Similarly, hypothesis tests on cointegrating vectors will typically be oversized if there is not an exact unit root. This paper uses a control variable approach where adding stationary covariates with certain properties to the model can result in asymptotic normal inference for prediction regressions and cointegration vector estimates in the presence of possibly non-unit root trending covariates. The properties required for this result are derived and discussed.