DIRECT MULTI-STEP ESTIMATION AND FORECASTING

Abstract.  This paper surveys the literature on multi-step forecasting when the model or the estimation method focuses directly on the link between the forecast origin and the horizon of interest. Among diverse contributions, we show how the current consensual concepts have emerged. We present an exhaustive overview of the existing results, including a conclusive review of the circumstances favourable to direct multi-step forecasting, namely different forms of non-stationarity and appropriate model design. We also provide a unifying framework which allows us to analyse the sources of forecast errors and hence of accuracy improvements from direct over iterated multi-step forecasting.     

Conditionally dependent strategies for multiple-step-ahead prediction in local learning

Computational intelligence approaches to multiple-step-ahead forecasting rely on either iterated one-step-ahead predictors or direct predictors. In both cases the predictions are obtained by means of multi-input single-output modeling techniques. This paper discusses the limitations of single-output approaches when the predictor is expected to return a long series of future values, and presents a multi-output approach to long term prediction. The motivation for this work is that, when predicting multiple steps ahead, the forecasted sequence should preserve the stochastic properties of the training series. However, this may not be the case, for instance in direct approaches where predictions for different horizons are produced independently. We discuss here a multi-output extension of conventional local modeling approaches, and present and compare three distinct criteria for performing conditionally dependent model selection. In order to assess the effectiveness of the different selection strategies, we carry out an extensive experimental session based on the 111 series in the NN5 competition.     

Comparing forecasts from fixed and variable coefficient models: The case of money demand

In this paper we introduce a class of tentatively plausible, fixed-coefficient models of money demand and evaluate their forecast performance. When these models are reestimated allowing all coefficients to vary over time, the forecasting performance improves dramatically. Aside from offering insights about improved methods of analyzing time series data, the most promising direct use for point estimates derived from time-varying coefficients is as an aid in calibrating proposed models of the kind discussed here.     

Efficiency of iterative estimators in the regression model with AR(1) disturbances

The mean square error approximation method of Nagar is applied to the iterated Prais-Winsten and (iterated) maximum likelihood estimators of regression coefficients in the model with AR(1) disturbances. Their mean square errors are found to equal that of the two-stage Prais-Winsten estimator at the second-order level of approximation.     

Housing market dynamics: Evidence of mean reversion and downward rigidity

House prices often exhibit serial correlation and mean reversion. Using two large panel datasets, this paper analyzes the price dynamics in two significantly different types of markets, cyclical (or volatile) and non-cyclical (or tame), by applying an autoregressive mean reversion (ARMR) model. Our results show that cyclical markets have larger AR coefficients than non-cyclical markets. As a result, house prices in cyclical markets tend to have larger price cycles. We also find that the upward periods have larger AR coefficients than the downward periods. This demonstrates that house prices are likely to overshoot the equilibrium in appreciating markets while experiencing downward rigidity during periods of decline. The model developed in this paper can produce a forecast with rich house price dynamics across markets. Our results can also be used to determine how house prices in overvalued markets will ultimately adjust.     

The performance of alternative forecasting methods for SETAR models

We compare a number of methods that have been proposed in the literature for obtaining h-step ahead minimum mean square error forecasts for self-exciting threshold autoregressive (SETAR) models. These forecasts are compared to those from an AR model. The comparison of forecasting methods is made using Monte Carlo simulation. The Monte-Carlo method of calculating SETAR forecasts is generally at least as good as that of the other methods we consider. An exception is when the disturbances in the SETAR model come from a highly asymmetric distribution, when a Bootstrap method is to be preferred.An empirical application calculates multi-period forecasts from a SETAR model of US gross national product using a number of the forecasting methods. We find that whether there are improvements in forecast performance relative to a linear AR model depends on the historical epoch we select, and whether forecasts are evaluated conditional on the regime the process was in at the time the forecast was made.     

Forecasting crude oil market volatility using variable selection and common factor

This paper aims to improve the predictability of aggregate oil market volatility with a substantially large macroeconomic database, including 127 macro variables. To this end, we use machine learning from both the variable selection (VS) and common factor (i.e., dimension reduction) perspectives. We first use the lasso, elastic net (ENet), and two conventional supervised learning approaches based on the significance level of predictors’ regression coefficients and the incremental R-square to select useful predictors relevant to forecasting oil market volatility. We then rely on the principal component analysis (PCA) to extract a common factor from the selected predictors. Finally, we augment the autoregression (AR) benchmark model by including the supervised PCA common index. Our empirical results show that the supervised PCA regression model can successfully predict oil market volatility both in-sample and out-of-sample. Also, the recommended models can yield forecasting gains in both statistical and economic perspectives. We further shed light on the nature of VS over time. In particular, option-implied volatility is always the most powerful predictor.     

Forecasting stock prices with commodity prices: New evidence from Feasible Quasi Generalized Least Squares (FQGLS) with non-linearities

The complexities in modern stock markets make it imperative to unravel the possible predictors of their future values. This paper thus provides insights into the predictability of stock prices of the BRICS countries with large dependence on commodities either for foreign exchange earnings or industrial while accounting for the role of asymmetries. Essentially, empirical evidence abound for the high volatility in world commodity markets, thus making us to determine if positive and negative changes in commodity prices predict stock prices differently. In addition, unlike the traditional forecast models, our choice of forecast models additionally addresses certain statistical features, including conditional heteroskedasticity, serial dependence, persistence and endogeneity, inherent in the predictors, which have the potential of causing estimation bias. In all, we find evidence in favour of the ability of commodity prices to predict stock prices of Brazil, Russia and South Africa. Also, both the in-sample and out-of-sample forecast performances of the predicted models support asymmetries in a number of commodity prices in each of these three countries. Our results are robust to different data samples and forecast horizons.     

Assessing the historical role of credit: Business cycles,financial crises and the legacy of Charles S. Peirce

This paper provides a historical overview of financial crises and their origins. The objective is to discuss a few of the modern statistical methods that can be used to evaluate predictors of these rare events. The problem involves the prediction of binary events, and therefore fits modern statistical learning, signal processing theory, and classification methods. The discussion also emphasizes the need for statistics and computational techniques to be supplemented with economics. The success of a forecast in this environment hinges on the economic consequences of the actions taken as a result of the forecast, rather than on typical statistical metrics of prediction accuracy.     

Complete subset regressions

This paper proposes a new method for combining forecasts based on complete subset regressions. For a given set of potential predictor variables we combine forecasts from all possible linear regression models that keep the number of predictors fixed. We explore how the choice of model complexity, as measured by the number of included predictor variables, can be used to trade off the bias and variance of the forecast errors, generating a setup akin to the efficient frontier known from modern portfolio theory. In an application to predictability of stock returns, we find that combinations of subset regressions can produce more accurate forecasts than conventional approaches based on equal-weighted forecasts (which fail to account for the dimensionality of the underlying models), combinations of univariate forecasts, or forecasts generated by methods such as bagging, ridge regression or Bayesian Model Averaging.     

Evaluating consumer sentiments as predictors of UK household consumption behavior: Are they accurate and useful?

This paper investigates empirically whether consumer sentiments indices, based on surveys complied by GfK, forecast household consumption types for the UK. Firstly, we use a quantitative equation approach to assess whether the indices are able to forecast household consumption growth in addition to traditional variables, which are included as control variables. Subsequently, using qualitative directional analysis, we investigate whether the indices are accurate and useful predictors as well. We find that, broadly speaking, both the headline, or aggregate, and the major purchasing indices have some predictive powers in addition to the control variables and are also directionally accurate and useful.     

Combination schemes for turning point predictions

We propose new forecast combination schemes for predicting turning points of business cycles. The proposed combination schemes are based on the forecasting performances of a given set of models with the aim to provide better turning point predictions. In particular, we consider predictions generated by autoregressive (AR) and Markov-switching AR models, which are commonly used for business cycle analysis. In order to account for parameter uncertainty we consider a Bayesian approach for both estimation and prediction and compare, in terms of statistical accuracy, the individual models and the combined turning point predictions for the United States and the Euro area business cycles.     

An empirical investigation of direct and iterated multistep conditional forecasts

When constructing unconditional point forecasts, both direct and iterated multistep (DMS and IMS) approaches are common. However, in the context of producing conditional forecasts, IMS approaches based on vector autoregressions are far more common than simpler DMS models. This is despite the fact that there are theoretical reasons to believe that DMS models are more robust to misspecification than are IMS models. In the context of unconditional forecasts, Marcellino et al. (Journal of Econometrics, 2006, 135, 499–526) investigate the empirical relevance of these theories. In this paper, we extend that work to conditional forecasts. We do so based on linear bivariate and trivariate models estimated using a large dataset of macroeconomic time series. Over comparable samples, our results reinforce those in Marcellino et al.: the IMS approach is typically a bit better than DMS with significant improvements only at longer horizons. In contrast, when we focus on the Great Moderation sample we find a marked improvement in the DMS approach relative to IMS. The distinction is particularly clear when we forecast nominal rather than real variables where the relative gains can be substantial.     

Expert opinion versus expertise in forecasting

Expert opinion is an opinion given by an expert, and it can have significant value in forecasting key policy variables in economics and finance. Expert forecasts can either be expert opinions, or forecasts based on an econometric model. An expert forecast that is based on an econometric model is replicable, and can be defined as a replicable expert forecast (REF), whereas an expert opinion that is not based on an econometric model can be defined as a non-replicable expert forecast (Non-REF). Both REF and Non-REF may be made available by an expert regarding a policy variable of interest. In this paper, we develop a model to generate REF, and compare REF with Non-REF. A method is presented to compare REF and Non-REF using efficient estimation methods, and a direct test of expertise on expert opinion is given. The latter serves the purpose of investigating whether expert adjustment improves the model-based forecasts. Illustrations for forecasting pharmaceutical stock keeping unit (SKUs), where the econometric model is of (variations of) the autoregressive integrated moving average model (ARIMA) type, show the relevance of the new methodology proposed in the paper. In particular, experts possess significant expertise, and expert forecasts are significant in explaining actual sales.     

Beyond point forecasting: Evaluation of alternative prediction intervals for tourist arrivals

This paper evaluates the performances of prediction intervals generated from alternative time series models, in the context of tourism forecasting. The forecasting methods considered include the autoregressive (AR) model, the AR model using the bias-corrected bootstrap, seasonal ARIMA models, innovations state space models for exponential smoothing, and Harvey’s structural time series models. We use thirteen monthly time series for the number of tourist arrivals to Hong Kong and Australia. The mean coverage rates and widths of the alternative prediction intervals are evaluated in an empirical setting. It is found that all models produce satisfactory prediction intervals, except for the autoregressive model. In particular, those based on the bias-corrected bootstrap perform best in general, providing tight intervals with accurate coverage rates, especially when the forecast horizon is long.     

Forecasting with Medium and Large Bayesian VARS

This paper is motivated by the recent interest in the use of Bayesian VARs for forecasting, even in cases where the number of dependent variables is large. In such cases factor methods have been traditionally used, but recent work using a particular prior suggests that Bayesian VAR methods can forecast better. In this paper, we consider a range of alternative priors which have been used with small VARs, discuss the issues which arise when they are used with medium and large VARs and examine their forecast performance using a US macroeconomic dataset containing 168 variables. We find that Bayesian VARs do tend to forecast better than factor methods and provide an extensive comparison of the strengths and weaknesses of various approaches. Typically, we find that the simple Minnesota prior forecasts well in medium and large VARs, which makes this prior attractive relative to computationally more demanding alternatives. Our empirical results show the importance of using forecast metrics based on the entire predictive density, instead of relying solely on those based on point forecasts. Copyright © 2011 John Wiley & Sons, Ltd.     

How accurate are government forecasts of economic fundamentals? The case of Taiwan

A government’s ability to forecast key economic fundamentals accurately can affect business confidence, consumer sentiment, and foreign direct investment, among others. A government forecast based on an econometric model is replicable, whereas one that is not fully based on an econometric model is non-replicable. Governments typically provide non-replicable forecasts (or expert forecasts) of economic fundamentals, such as the inflation rate and real GDP growth rate.In this paper, we develop a methodology for evaluating non-replicable forecasts. We argue that in order to do so, one needs to retrieve from the non-replicable forecast its replicable component, and that it is the difference in accuracy between these two that matters. An empirical example to forecast economic fundamentals for Taiwan shows the relevance of the proposed methodological approach. Our main finding is that the undocumented knowledge of the Taiwanese government reduces forecast errors substantially.     

Macroeconomic forecasting for Australia using a large number of predictors

A popular approach to forecasting macroeconomic variables is to utilize a large number of predictors. Several regularization and shrinkage methods can be used to exploit such high-dimensional datasets, and have been shown to improve forecast accuracy for the US economy. To assess whether similar results hold for economies with different characteristics, an Australian dataset containing observations on 151 aggregate and disaggregate economic series as well as 185 international variables, is introduced. An extensive empirical study is carried out investigating forecasts at different horizons, using a variety of methods and with information sets containing an increasing number of predictors. In contrast to other countries the results show that it is difficult to forecast Australian key macroeconomic variables more accurately than some simple benchmarks. In line with other studies we also find that there is little to no improvement in forecast accuracy when the number of predictors is expanded beyond 20–40 variables and international factors do not seem to help.     

Semiparametric quantile averaging in the presence of high-dimensional predictors

The paper proposes a method for forecasting conditional quantiles. In practice, one often does not know the “true” structure of the underlying conditional quantile function, and in addition, we may have a large number of predictors. Focusing on such cases, we introduce a flexible and practical framework based on penalized high-dimensional quantile averaging. In addition to prediction, we show that the proposed method can also serve as a predictor selector. We conduct extensive simulation experiments to asses its prediction and variable selection performances for nonlinear and linear time series model designs. In terms of predictor selection, the approach tends to select the true set of predictors with minimal false positives. With respect to prediction accuracy, the method competes well even with the benchmark/oracle methods that know one or more aspects of the underlying quantile regression model. We further illustrate the merit of the proposed method by providing an application to the out-of-sample forecasting of U.S. core inflation using a large set of monthly macroeconomic variables based on FRED-MD database. The application offers several empirical findings.     

Evaluating density forecasts: forecast combinations,model mixtures,calibration and sharpness

This paper reviews current density forecast evaluation procedures, and considers a proposal that such procedures be augmented by an assessment of ‘sharpness’. This was motivated by an example in which some standard evaluation procedures using probability integral transforms cannot distinguish the ideal forecast from several competing forecasts. We show that this example has some unrealistic features from a time series forecasting perspective, and so provides insecure foundations for the argument that existing calibration procedures are inadequate in practice. Our alternative, more realistic example shows how relevant statistical methods, including information‐based methods, provide the required discrimination between competing forecasts. We introduce a new test of density forecast efficiency. Copyright © 2010 John Wiley & Sons, Ltd.