Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination

In this paper, we examine the forecast accuracy of linear autoregressive, smooth transition autoregressive (STAR), and neural network (NN) time series models for 47 monthly macroeconomic variables of the G7 economies. Unlike previous studies that typically consider multiple but fixed model specifications, we use a single but dynamic specification for each model class. The point forecast results indicate that the STAR model generally outperforms linear autoregressive models. It also improves upon several fixed STAR models, demonstrating that careful specification of nonlinear time series models is of crucial importance. The results for neural network models are mixed in the sense that at long forecast horizons, an NN model obtained using Bayesian regularization produces more accurate forecasts than a corresponding model specified using the specific-to-general approach. Reasons for this outcome are discussed.     

Modelling and Forecasting Fiscal Variables for the Euro Area*

In this paper, we assess the possibility of producing unbiased forecasts for fiscal variables in the Euro area by comparing a set of procedures that rely on different information sets and econometric techniques. In particular, we consider autoregressive moving average models, Vector autoregressions, small‐scale semistructural models at the national and Euro area level, institutional forecasts (Organization for Economic Co‐operation and Development), and pooling. Our small‐scale models are characterized by the joint modelling of fiscal and monetary policy using simple rules, combined with equations for the evolution of all the relevant fundamentals for the Maastricht Treaty and the Stability and Growth Pact. We rank models on the basis of their forecasting performance using the mean square and mean absolute error criteria at different horizons. Overall, simple time‐series methods and pooling work well and are able to deliver unbiased forecasts, or slightly upward‐biased forecast for the debt–GDP dynamics. This result is mostly due to the short sample available, the robustness of simple methods to structural breaks, and to the difficulty of modelling the joint behaviour of several variables in a period of substantial institutional and economic changes. A bootstrap experiment highlights that, even when the data are generated using the estimated small‐scale multi‐country model, simple time‐series models can produce more accurate forecasts, because of their parsimonious specification.     

Forecasting United States–Asia international message telephone service

This study compares forecasts of US international message telephone service (IMTS) traffic using several relative mean squared error statistics. The forecasts are obtained from time-series extrapolation, univariate autoregressive integrated moving average (ARIMA), error correction and vector autoregressive models. The models are estimated on annual US IMTS outgoing traffic data for six US–Asia bilateral markets for the period 1964 to 1993. No single approach provides best forecasts. However, forecast evaluation statistics indicate that econometric models generally outperform the alternatives.     

A new marked point process model for the federal funds rate target: Methodology and forecast evaluation

Forecasts of key interest rates set by central banks are of paramount concern for investors and policy makers. Recently it has been shown that forecasts of the federal funds rate target, the most anticipated indicator of the Federal Reserve Bank's monetary policy stance, can be improved considerably when its evolution is modeled as a marked point process (MPP). This is due to the fact that target changes occur in discrete time with discrete increments, have an autoregressive nature and are usually in the same direction. We propose a model which is able to account for these dynamic features of the data. In particular, we combine Hamilton and Jordà's [2002. A model for the federal funds rate target. Journal of Political Economy 110(5), 1135–1167] autoregressive conditional hazard (ACH) and Russell and Engle's [2005. A discrete-state continuous-time model of financial transactions prices and times: the autoregressive conditional multinomial-autoregressive conditional duration model. Journal of Business and Economic Statistics 23(2), 166 – 180] autoregressive conditional multinomial (ACM) model. The paper also puts forth a methodology to evaluate probability function forecasts of MPP models. By improving goodness of fit and point forecasts of the target, the ACH–ACM qualifies as a sensible modeling framework. Furthermore, our results show that MPP models deliver useful probability function forecasts at short and medium term horizons.     

Macroeconomic forecast accuracy in a data‐rich environment

The performance of six classes of models in forecasting different types of economic series is evaluated in an extensive pseudo out‐of‐sample exercise. One of these forecasting models, regularized data‐rich model averaging (RDRMA), is new in the literature. The findings can be summarized in four points. First, RDRMA is difficult to beat in general and generates the best forecasts for real variables. This performance is attributed to the combination of regularization and model averaging, and it confirms that a smart handling of large data sets can lead to substantial improvements over univariate approaches. Second, the ARMA(1,1) model emerges as the best to forecast inflation changes in the short run, while RDRMA dominates at longer horizons. Third, the returns on the S&P 500 index are predictable by RDRMA at short horizons. Finally, the forecast accuracy and the optimal structure of the forecasting equations are quite unstable over time.     

Determination of Long‐run and Short‐run Dynamics in EC‐VARMA Models via Canonical Correlations

This article studies a simple, coherent approach for identifying and estimating error‐correcting vector autoregressive moving average (EC‐VARMA) models. Canonical correlation analysis is implemented for both determining the cointegrating rank, using a strongly consistent method, and identifying the short‐run VARMA dynamics, using the scalar component methodology. Finite‐sample performance is evaluated via Monte Carlo simulations and the approach is applied to modelling and forecasting US interest rates. The results reveal that EC‐VARMA models generate significantly more accurate out‐of‐sample forecasts than vector error correction models (VECMs), especially for short horizons. Copyright © 2015 John Wiley & Sons, Ltd.     

Forecasts of US short-term interest rates: A flexible forecast combination approach

This paper develops a flexible approach to combine forecasts of future spot rates with forecasts from time-series models or macroeconomic variables. We find empirical evidence that, accounting for both regimes in interest rate dynamics, and combining forecasts from different models, helps improve the out-of-sample forecasting performance for US short-term rates. Imposing restrictions from the expectations hypothesis on the forecasting model are found to help at long forecasting horizons.     

Experience curve models in the electricity supply industry

There has been much controversy over the use of the Experience Curve for forecasting purposes. The Experience Curve model has been criticised both on theoretical grounds and because of the practical problems of using it. An alternative model of experience effects due to Towill has certain attractions from the standpoint of theory. However, a rather deeper question is whether experience curve type models produce superior forecasts to those derived using extrapolative techniques.This paper examines these questions in the context of three time series taken from the electricity supply industry, viz: average thermal efficiency; works costs; and price of electricity. The two latter series require price deflation. Both the implied GDP consumption deflator, and a wholesale price index for fuel and electricity were used for this purpose. It is argued that because of the absence of substitutes and of the effects of competition, along with the high quality of data available on the electricity supply industry, these three series provide a favourable test of the experience curve approach to forecasting. The two experience curves performed on the whole markedly worse than the simpler extrapolative methods on the two financial series examined. For the average thermal efficiency series the Towill model and the Experience Curve model marginally outperformed the extrapolative methods.Overall, there was little support for using either the Experience Curve or Towill models. These are obviously more difficult to use than simple univariate models and do not provide significantly better forecasts. Moreover, the Towill model gave rise to considerable estimation and specification problems with the data used here.     

Assessing the uncertainty in central banks’ inflation outlooks

Recent research has found that macroeconomic survey forecasts of uncertainty exhibit several deficiencies, such as horizon-dependent biases and lower levels of accuracy than simple unconditional uncertainty forecasts. We examine the inflation uncertainty forecasts from the Bank of England, the Banco Central do Brasil, the Magyar Nemzeti Bank and the Sveriges Riksbank to assess whether central banks’ uncertainty forecasts might be subject to similar problems. We find that, while most central banks’ uncertainty forecasts also tend to be underconfident at short horizons and overconfident at longer horizons, they are mostly not significantly biased. Moreover, they tend to be at least as precise as unconditional uncertainty forecasts from two different approaches.     

Evaluating a three-dimensional panel of point forecasts: The Bank of England Survey of External Forecasters

This article provides a first analysis of the forecasts of inflation and GDP growth obtained from the Bank of England's Survey of External Forecasters, considering both the survey average forecasts published in the quarterly Inflation Report, and the individual survey responses, recently made available by the Bank. These comprise a conventional incomplete panel dataset, with an additional dimension arising from the collection of forecasts at several horizons; both point forecasts and density forecasts are collected. The inflation forecasts show good performance in tests of unbiasedness and efficiency, albeit over a relatively calm period for the UK economy, and there is considerable individual heterogeneity. For GDP growth, inaccurate real-time data and their subsequent revisions are seen to cause serious difficulties for forecast construction and evaluation, although the forecasts are again unbiased. There is evidence that some forecasters have asymmetric loss functions.     

Forecasting exchange rate volatility using high-frequency data: Is the euro different?

We assess the performances of alternative procedures for forecasting the daily volatility of the euro’s bilateral exchange rates using 15 min data. We use realized volatility and traditional time series volatility models. Our results indicate that using high-frequency data and considering their long memory dimension enhances the performance of volatility forecasts significantly. We find that the intraday FIGARCH model and the ARFIMA model outperform other traditional models for all exchange rate series.     

Do precious metal prices help in forecasting South African inflation?

In this paper we test whether the key metals prices of gold and platinum significantly improve inflation forecasts for the South African economy. We also test whether controlling for conditional correlations in a dynamic setup, using bivariate Bayesian-Dynamic Conditional Correlation (B-DCC) models, improves inflation forecasts. To achieve this we compare out-of-sample forecast estimates of the B-DCC model to Random Walk, Autoregressive and Bayesian VAR models. We find that for both the BVAR and BDCC models, improving point forecasts of the Autoregressive model of inflation remains an elusive exercise. This, we argue, is of less importance relative to the more informative density forecasts. For this we find improved forecasts of inflation for the B-DCC models at all forecasting horizons tested. We thus conclude that including metals price series as inputs to inflation models leads to improved density forecasts, while controlling for the dynamic relationship between the included price series and inflation similarly leads to significantly improved density forecasts.     

Evaluating forecasts of political conflict dynamics

There is considerable interest today in the forecasting of conflict dynamics. Commonly, the root mean square error and other point metrics are used to evaluate the forecasts from such models. However, conflict processes are non-linear, so these point metrics often do not produce adequate evaluations of the calibration and sharpness of the forecast models. Forecast density evaluation improves the model evaluation. We review tools for density evaluation, including continuous rank probability scores, verification rank histograms, and sharpness plots. The usefulness of these tools for evaluating conflict forecasting models is explained. We illustrate this, first, in a comparison of several time series models’ forecasts of simulated data from a Markov-switching process, and second, in a comparison of several models’ abilities to forecast conflict dynamics in the Cross Straits. These applications show the pitfalls of relying on point metrics alone for evaluating the quality of conflict forecasting models. As in other fields, it is more useful to employ a suite of tools. A non-linear vector autoregressive model emerges as the model which is best able to forecast conflict dynamics between China and Taiwan.     

Short-term forecasting of the coronavirus pandemic

We have been publishing real-time forecasts of confirmed cases and deaths from coronavirus disease 2019 (COVID-19) since mid-March 2020 (published at www.doornik.com/COVID-19). These forecasts are short-term statistical extrapolations of past and current data. They assume that the underlying trend is informative regarding short-term developments but without requiring other assumptions about how the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus is spreading, or whether preventative policies are effective. Thus, they are complementary to the forecasts obtained from epidemiological models.The forecasts are based on extracting trends from windows of data using machine learning and then computing the forecasts by applying some constraints to the flexible extracted trend. These methods have been applied previously to various other time series data and they performed well. They have also proved effective in the COVID-19 setting where they provided better forecasts than some epidemiological models in the earlier stages of the pandemic.     

EVALUATING MACROECONOMIC FORECASTS: A CONCISE REVIEW OF SOME RECENT DEVELOPMENTS

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.     

Online hierarchical forecasting for power consumption data

This paper proposes a three-step approach to forecasting time series of electricity consumption at different levels of household aggregation. These series are linked by hierarchical constraints—global consumption is the sum of regional consumption, for example. First, benchmark forecasts are generated for all series using generalized additive models. Second, for each series, the aggregation algorithm ML-Poly, introduced by Gaillard, Stoltz, and van Erven in 2014, finds an optimal linear combination of the benchmarks. Finally, the forecasts are projected onto a coherent subspace to ensure that the final forecasts satisfy the hierarchical constraints. By minimizing a regret criterion, we show that the aggregation and projection steps improve the root mean square error of the forecasts. Our approach is tested on household electricity consumption data; experimental results suggest that successive aggregation and projection steps improve the benchmark forecasts at different levels of household aggregation.     

Combining forecasts: Performance and coherence

There is general agreement in many forecasting contexts that combining individual predictions leads to better final forecasts. However, the relative error reduction in a combined forecast depends upon the extent to which the component forecasts contain unique/independent information. Unfortunately, obtaining independent predictions is difficult in many situations, as these forecasts may be based on similar statistical models and/or overlapping information. The current study addresses this problem by incorporating a measure of coherence into an analytic evaluation framework so that the degree of independence between sets of forecasts can be identified easily. The framework also decomposes the performance and coherence measures in order to illustrate the underlying aspects that are responsible for error reduction. The framework is demonstrated using UK retail prices index inflation forecasts for the period 1998–2014, and implications for forecast users are discussed.     

Assessing forecast performance in a cointegrated system

This paper examines the forecast performance of a cointegrated system relative to the forecast performance of a comparable VAR that fails to recognize that the system is characterized by cointegration. The cointegrated system we examine is composed of three vectors, a money demand representation, a Fisher equation, and a risk premium captured by an interest rate differential. The forecasts produced by the vector error correction model (VECM) associated with this system are compared with those obtained from a corresponding differenced vector autoregression, (DVAR) as well as a vector autoregression based upon the levels of the data (LVAR). Forecast evaluation is conducted using both the ‘full-system’ criterion proposed by Clements and Hendry (1993) and by comparing forecast performance for specific variables. Overall our findings suggest that selective forecast performance improvement (especially at long forecast horizons) may be observed by incorporating knowledge of cointegration rank. Our general conclusion is that when the advantage of incorporating cointegration appears, it is generally at longer forecast horizons. This is consistent with the predictions of Engle and Yoo (1987). But we also find, consistent with Clements and Hendry (1995) that relative gain in forecast performance clearly depends upon the chosen data transformation.     

One model and various experts: Evaluating Dutch macroeconomic forecasts

The Netherlands Bureau for Economic Policy Analysis (CPB) uses a large macroeconomic model to create forecasts of various important macroeconomic variables. The outcomes of this model are usually filtered by experts, and it is the expert forecasts that are made available to the general public. In this paper we re-create the model forecasts for the period 1997-2008 and compare the expert forecasts with the pure model forecasts. Our key findings from the first time that this unique database has been analyzed are that (i) experts adjust upwards more often; (ii) expert adjustments are not autocorrelated, but their sizes do depend on the value of the model forecast; (iii) the CPB model forecasts are biased for a range of variables, but (iv) at the same time, the associated expert forecasts are more often unbiased; and that (v) expert forecasts are far more accurate than the model forecasts, particularly when the forecast horizon is short. In summary, the final CPB forecasts de-bias the model forecasts and lead to higher accuracies than the initial model forecasts.