A comprehensive evaluation of macroeconomic forecasting methods

We employ datasets for seven developed economies and consider four classes of multivariate forecasting models in order to extend and enhance the empirical evidence in the macroeconomic forecasting literature. The evaluation considers forecasting horizons of between one quarter and two years ahead. We find that the structural model, a medium-sized DSGE model, provides accurate long-horizon US and UK inflation forecasts. We strike a balance between being comprehensive and producing clear messages by applying meta-analysis regressions to 2,976 relative accuracy comparisons that vary with the forecasting horizon, country, model class and specification, number of predictors, and evaluation period. For point and density forecasting of GDP growth and inflation, we find that models with large numbers of predictors do not outperform models with 13–14 hand-picked predictors. Factor-augmented models and equal-weighted combinations of single-predictor mixed-data sampling regressions are a better choice for dealing with large numbers of predictors than Bayesian VARs.     

A functional time series analysis of forward curves derived from commodity futures

We study forward curves formed from commodity futures prices listed on the Standard and Poor’s-Goldman Sachs Commodities Index (S&P GSCI) using recently developed tools in functional time series analysis. Functional tests for stationarity and serial correlation suggest that log-differenced forward curves may be generally considered as stationary and conditionally heteroscedastic sequences of functions. Several functional methods for forecasting forward curves that more accurately reflect the time to expiry of contracts are developed, and we found that these typically outperformed their multivariate counterparts, with the best among them using the method of predictive factors introduced by Kargin and Onatski (2008).     

Exponentially weighted information criteria for selecting among forecasting models

Information criteria (IC) are often used to decide between forecasting models. Commonly used criteria include Akaike's IC and Schwarz's Bayesian IC. They involve the sum of two terms: the model's log likelihood and a penalty for the number of model parameters. The likelihood is calculated with equal weight being given to all observations. We propose that greater weight should be put on more recent observations in order to reflect more recent accuracy. This seems particularly pertinent when selecting among exponential smoothing methods, as they are based on an exponential weighting principle. In this paper, we use exponential weighting within the calculation of the log likelihood for the IC. Our empirical analysis uses supermarket sales and call centre arrivals data. The results show that basing model selection on the new exponentially weighted IC can outperform individual models and selection based on the standard IC.     

Macroeconomic forecasting using structural factor analysis

The use of a small number of underlying factors to summarize the information from a much larger set of information variables is one of the new frontiers in forecasting. In prior work, the estimated factors have not usually had a structural interpretation and the factors have not been chosen on a theoretical basis. In this paper we propose several variants of a general structural factor forecasting model, and use these to forecast certain key macroeconomic variables. We make the choice of factors more structurally meaningful by estimating factors from subsets of information variables, where these variables can be assigned to subsets on the basis of economic theory. We compare the forecasting performance of the structural factor forecasting model with that of a univariate AR model, a standard VAR model, and some non-structural factor forecasting models. The results suggest that our structural factor forecasting model performs significantly better in forecasting real activity variables, especially at short horizons.     

A SHARP model of bid–ask spread forecasts

This paper proposes an accurate, parsimonious and fast-to-estimate forecasting model for integer-valued time series with long memory and seasonality. The modelling is achieved through an autoregressive Poisson process with a predictable stochastic intensity that is determined by two factors: a seasonal intraday pattern and a heterogeneous autoregressive component. We call the model SHARP, which is an acronym for seasonal heterogeneous autoregressive Poisson. We also present a mixed-data sampling extension of the model, which adopts the historical information flow more efficiently and provides the best (among all the models considered) forecasting performances, empirically, for the bid–ask spreads of NYSE equity stocks. We conclude by showing how bid–ask spread forecasts based on the SHARP model can be exploited in order to reduce the total cost incurred by a trader who is willing to buy or sell a given amount of an equity stock.     

A calibration procedure for analyzing stock price dynamics in an agent-based framework

In this paper we introduce a calibration procedure for validating of agent based models. Starting from the well-known financial model of (Brock and Hommes, 1998), we show how an appropriate calibration enables the model to describe price time series. We formulate the calibration problem as a nonlinear constrained optimization that can be solved numerically via a gradient-based method. The calibration results show that the simplest version of the Brock and Hommes model, with two trader types, fundamentalists and trend-followers, replicates nicely the price series of four different markets indices: the S&P 500, the Euro Stoxx 50, the Nikkei 225 and the CSI 300. We show how the parameter values of the calibrated model are important in interpreting the trader behavior in the different markets investigated. These parameters are then used for price forecasting. To further improve the forecasting, we modify our calibration approach by increasing the trader information set. Finally, we show how this new approach improves the model׳s ability to predict market prices.     

Forecast on silver futures linked with structural breaks and day-of-the-week effect

Silver future is crucial to global financial markets. However, the existing literature rarely considers the impacts of structural breaks and day-of-the-week effect simultaneously on the volatility of silver future price. Based on heterogeneous autoregressive (HAR) theory, we establish six new type heterogeneous autoregressive (HAR) models by incorporating structural breaks and day-of-the-week effect to forecast the volatility. The empirical results indicate that new models’ accuracy is better than the original HAR model. We find that structural breaks and the day-of-the-week effect contain much forecasting information on silver forecasting. In addition, structural breaks have a positive effect on the silver futures’ volatility. Day-of-the-week effect has a significantly negative influence on silver futures’ price volatility, especially in the mid-term and the long-term. Our works is the first to combine the structural breaks and day-of-the-week effect to identify more market information. This paper provides a better forecasting method to predict silver future volatility.     

Forecasting temperature to price CME temperature derivatives

This paper seeks to forecast temperatures in US cities in order to price temperature derivatives on the Chicago Mercantile Exchange (CME). The CME defines the average daily temperature underlying its contracts as the average of the maximum and minimum daily temperatures, yet all published work on temperature forecasting for pricing purposes has ignored this peculiar definition of the average and sought to model the average temperature directly. This paper is the first to look at the average temperature forecasting problem as an analysis of extreme values. The theory of extreme values guides model selection for temperature maxima and minima, and a forecast distribution for the CME’s daily average temperature is found through convolution. While univariate time series AR-GARCH and regression models generally yield superior point forecasts of temperatures, our extreme-value-based model consistently outperforms these models in density forecasting, the most important risk management tool.     

Stochastic volatility models with ARMA innovations: An application to G7 inflation forecasts

We introduce a new class of stochastic volatility models with autoregressive moving average (ARMA) innovations. The conditional mean process has a flexible form that can accommodate both a state space representation and a conventional dynamic regression. The ARMA component introduces serial dependence, which results in standard Kalman filter techniques not being directly applicable. To overcome this hurdle, we develop an efficient posterior simulator that builds on recently developed precision-based algorithms. We assess the usefulness of these new models in an inflation forecasting exercise across all G7 economies. We find that the new models generally provide competitive point and density forecasts compared to standard benchmarks, and are especially useful for Canada, France, Italy, and the U.S.     

Probabilistic forecasting of industrial electricity load with regime switching behavior

This paper suggests a novel inhomogeneous Markov switching approach for the probabilistic forecasting of industrial companies’ electricity loads, for which the load switches at random times between production and standby regimes. The model that we propose describes the transitions between the regimes using a hidden Markov chain with time-varying transition probabilities that depend on calendar variables. We model the demand during the production regime using an autoregressive moving-average (ARMA) process with seasonal patterns, whereas we use a much simpler model for the standby regime in order to reduce the complexity. The maximum likelihood estimation of the parameters is implemented using a differential evolution algorithm. Using the continuous ranked probability score (CRPS) to evaluate the goodness-of-fit of our model for probabilistic forecasting, it is shown that this model often outperforms classical additive time series models, as well as homogeneous Markov switching models. We also propose a simple procedure for classifying load profiles into those with and without regime-switching behaviors.     

A note on forecasting international tourism demand in Spain

This paper considers the extent to which price and income proxy variables help in forecasting tourist demand in Spain. Contrary to some recent studies, we found that the inputs' contribution in terms of fitting and forecasting is nil when compared with alternative univariate models. Whether these findings are the results of the restrictions embedded in building the proxy inputs or in a poor specification of the dynamics of these models remains to be seen. We also contend that when dealing with medium, long-term forecasting comparisons, the use of the traditional aggregate accuracy measures like RMSE and MAPE help very little in discriminating among competing models. In these situations, predicted annual growth rates may be a better alternative.     

Variable selection, estimation and inference for multi-period forecasting problems

This paper conducts a broad-based comparison of iterated and direct multi-period forecasting approaches applied to both univariate and multivariate models in the form of parsimonious factor-augmented vector autoregressions. To account for serial correlation in the residuals of the multi-period direct forecasting models we propose a new SURE-based estimation method and modified Akaike information criteria for model selection. Empirical analysis of the 170 variables studied by Marcellino, Stock and Watson (2006) shows that information in factors helps improve forecasting performance for most types of economic variables although it can also lead to larger biases. It also shows that SURE estimation and finite-sample modifications to the Akaike information criterion can improve the performance of the direct multi-period forecasts.     

DSGE forecasts of the lost recovery

The years following the Great Recession were challenging for forecasters. Unlike other deep downturns, this recession was not followed by a swift recovery, but instead generated a sizable and persistent output gap that was not accompanied by deflation as a traditional Phillips curve relationship would have predicted. Moreover, the zero lower bound and unconventional monetary policy generated an unprecedented policy environment. We document the actual real-time forecasting performance of the New York Fed dynamic stochastic general equilibrium (DSGE) model during this period and explain the results using the pseudo real-time forecasting performance results from a battery of DSGE models. We find the New York Fed DSGE model’s forecasting accuracy to be comparable to that of private forecasters, and notably better for output growth than the median forecasts from the FOMC’s Summary of Economic Projections. The model’s financial frictions were key in obtaining these results, as they implied a slow recovery following the financial crisis.     

Multivariate semi-nonparametric distributions with dynamic conditional correlations

This paper generalizes the Dynamic Conditional Correlation (DCC) model of Engle (2002), incorporating a flexible non-Gaussian distribution based on Gram-Charlier expansions. The resulting semi-nonparametric-DCC (SNP-DCC) model allows estimation in two stages and deals with the negativity problem which is inherent in truncated SNP densities. We test the performance of a SNP-DCC model with respect to the (Gaussian)-DCC through an empirical application of density forecasting for portfolio returns. Our results show that the proposed multivariate model provides a better in-sample fit and forecast of the portfolio returns distribution, and thus is useful for financial risk forecasting and evaluation.     

Forecasting the UK economy with a medium-scale Bayesian VAR

We estimate a Bayesian VAR (BVAR) for the UK economy and assess its performance in forecasting GDP growth and CPI inflation in real time relative to forecasts from COMPASS, the Bank of England’s DSGE model, and other benchmarks. We find that the BVAR outperformed COMPASS when forecasting both GDP and its expenditure components. In contrast, their performances when forecasting CPI were similar. We also find that the BVAR density forecasts outperformed those of COMPASS, despite under-predicting inflation at most forecast horizons. Both models over-predicted GDP growth at all forecast horizons, but the issue was less pronounced in the BVAR. The BVAR’s point and density forecast performances are also comparable to those of a Bank of England in-house statistical suite for both GDP and CPI inflation, as well as to the official Inflation Report projections. Our results are broadly consistent with the findings of similar studies for other advanced economies.     

Forecasting the UK economy: Alternative forecasting methodologies and the role of off-model information

How did DSGE model forecasts perform before, during and after the financial crisis, and what type of off-model information can improve the forecast accuracy? We tackle these questions by assessing the real-time forecast performance of a large DSGE model relative to statistical and judgmental benchmarks over the period from 2000 to 2013. The forecasting performances of all methods deteriorate substantially following the financial crisis. That is particularly evident for the DSGE model’s GDP forecasts, but augmenting the model with a measure of survey expectations made its GDP forecasts more accurate, which supports the idea that timely off-model information is particularly useful in times of financial distress.     

Analysts' Forecasts of German Firms' Earnings: a Comparative Analysis

This paper examines analysts' forecasts of the annual earnings per share of German firms over the period of February 1987 to December 1995. The German case is particularly interesting as the accounting and institutional structures vary from those in more thoroughly researched markets such as the U.S. or U.K. The paper therefore considers the features of the German forecasting environment which distinguish it from the Anglo-American model, and whether these might be reflected in forecasting performance. The results for Germany show that the accuracy of analysts' forecasts improves as the forecast horizon shortens, are less accurate than a naive prediction model over longer horizons, and contain a positive bias. When the results for Germany are contrasted with the results for the U.K., as reported in a recent paper, they are found to be a little less accurate but the positive bias is greater in U.K. forecasts. Taken overall the forecasting process in Germany appears to be less efficient than in the U.K., but this may be due to the distinct features of the German forecasting environment.     

Improving forecasting for telemarketing centers by ARIMA modeling with intervention

In this study we analyze existing and improved methods for forecasting incoming calls to telemarketing centers for the purposes of planning and budgeting. We analyze the use of additive and multiplicative versions of Holt–Winters (HW) exponentially weighted moving average models and compare it to Box–Jenkins (ARIMA) modeling with intervention analysis. We determine the forecasting accuracy of HW and ARIMA models for samples of telemarketing data. Although there is much evidence in recent literature that “simple models” such as Holt–Winters perform as well as or better than more complex models, we find that ARIMA models with intervention analysis perform better for the time series studied.     

Forecasting with vector autoregressive (VAR) models subject to business cycle restrictions

In the last decade VAR models have become a widely-used tool for forecasting macroeconomic time series. To improve the out-of-sample forecasting accuracy of these models, Bayesian random-walk prior restrictions are often imposed on VAR model parameters. This paper focuses on whether placing an alternative type of restriction on the parameters of unrestricted VAR models improves the out-of-sample forecasting performance of these models. The type of restriction analyzed here is based on the business cycle characteristics of U.S. macroeconomic data, and in particular, requires that the dynamic behavior of the restricted VAR model mimic the business cycle characteristics of historical data. The question posed in this paper is: would a VAR model, estimated subject to the restriction that the cyclical characteristics of simulated data from the model “match up” with the business cycle characteristics of U.S. data, generate more accurate out-of-sample forecasts than unrestricted or Bayesian VAR models?     

Bayesian time series analysis of periodic behaviour and spectral structure

Analysis, model selection and forecasting in univariate time series models can be routinely carried out for models in which the model order is relatively small. Under an ARMA assumption, classical estimation, model selection and forecasting can be routinely implemented with the Box–Jenkins time domain representation. However, this approach becomes at best prohibitive and at worst impossible when the model order is high. In particular, the standard assumption of stationarity imposes constraints on the parameter space that are increasingly complex. One solution within the pure AR domain is the latent root factorization in which the characteristic polynomial of the AR model is factorized in the complex domain, and where inference questions of interest and their solution are expressed in terms of the implied (reciprocal) complex roots; by allowing for unit roots, this factorization can identify any sustained periodic components. In this paper, as an alternative to identifying periodic behaviour, we concentrate on frequency domain inference and parameterize the spectrum in terms of the reciprocal roots, and, in addition, incorporate Gegenbauer components. We discuss a Bayesian solution to the various inference problems associated with model selection involving a Markov chain Monte Carlo (MCMC) analysis. One key development presented is a new approach to forecasting that utilizes a Metropolis step to obtain predictions in the time domain even though inference is being carried out in the frequency domain. This approach provides a more complete Bayesian solution to forecasting for ARMA models than the traditional approach that truncates the infinite AR representation, and extends naturally to Gegenbauer ARMA and fractionally differenced models.