LoMEF: A framework to produce local explanations for global model time series forecasts

Global forecasting models (GFMs) that are trained across a set of multiple time series have shown superior results in many forecasting competitions and real-world applications compared with univariate forecasting approaches. One aspect of the popularity of statistical forecasting models such as ETS and ARIMA is their relative simplicity and interpretability (in terms of relevant lags, trend, seasonality, and other attributes), while GFMs typically lack interpretability, especially relating to particular time series. This reduces the trust and confidence of stakeholders when making decisions based on the forecasts without being able to understand the predictions. To mitigate this problem, we propose a novel local model-agnostic interpretability approach to explain the forecasts from GFMs. We train simpler univariate surrogate models that are considered interpretable (e.g., ETS) on the predictions of the GFM on samples within a neighbourhood that we obtain through bootstrapping, or straightforwardly as the one-step-ahead global black-box model forecasts of the time series which needs to be explained. After, we evaluate the explanations for the forecasts of the global models in both qualitative and quantitative aspects such as accuracy, fidelity, stability, and comprehensibility, and are able to show the benefits of our approach.     

An accurate and fully-automated ensemble model for weekly time series forecasting

Many businesses and industries require accurate forecasts for weekly time series nowadays. However, the forecasting literature does not currently provide easy-to-use, automatic, reproducible and accurate approaches dedicated to this task. We propose a forecasting method in this domain to fill this gap, leveraging state-of-the-art forecasting techniques, such as forecast combination, meta-learning, and global modelling. We consider different meta-learning architectures, algorithms, and base model pools. Based on all considered model variants, we propose to use a stacking approach with lasso regression which optimally combines the forecasts of four base models: a global Recurrent Neural Network (RNN) model, Theta, Trigonometric Box–Cox ARMA Trend Seasonal (TBATS), and Dynamic Harmonic Regression ARIMA (DHR-ARIMA), as it shows the overall best performance across seven experimental weekly datasets on four evaluation metrics. Our proposed method also consistently outperforms a set of benchmarks and state-of-the-art weekly forecasting models by a considerable margin with statistical significance. Our method can produce the most accurate forecasts, in terms of mean sMAPE, for the M4 weekly dataset among all benchmarks and all original competition participants.     

Distributed ARIMA models for ultra-long time series

Providing forecasts for ultra-long time series plays a vital role in various activities, such as investment decisions, industrial production arrangements, and farm management. This paper develops a novel distributed forecasting framework to tackle the challenges of forecasting ultra-long time series using the industry-standard MapReduce framework. The proposed model combination approach retains the local time dependency. It utilizes a straightforward splitting across samples to facilitate distributed forecasting by combining the local estimators of time series models delivered from worker nodes and minimizing a global loss function. Instead of unrealistically assuming the data generating process (DGP) of an ultra-long time series stays invariant, we only make assumptions on the DGP of subseries spanning shorter time periods. We investigate the performance of the proposed approach with AutoRegressive Integrated Moving Average (ARIMA) models using the real data application as well as numerical simulations. Our approach improves forecasting accuracy and computational efficiency in point forecasts and prediction intervals, especially for longer forecast horizons, compared to directly fitting the whole data with ARIMA models. Moreover, we explore some potential factors that may affect the forecasting performance of our approach.     

The tourism forecasting competition

We evaluate the performances of various methods for forecasting tourism data. The data used include 366 monthly series, 427 quarterly series and 518 annual series, all supplied to us by either tourism bodies or academics who had used them in previous tourism forecasting studies. The forecasting methods implemented in the competition are univariate and multivariate time series approaches, and econometric models. This forecasting competition differs from previous competitions in several ways: (i) we concentrate on tourism data only; (ii) we include approaches with explanatory variables; (iii) we evaluate the forecast interval coverage as well as the point forecast accuracy; (iv) we observe the effect of temporal aggregation on the forecasting accuracy; and (v) we consider the mean absolute scaled error as an alternative forecasting accuracy measure. We find that pure time series approaches provide more accurate forecasts for tourism data than models with explanatory variables. For seasonal data we implement three fully automated pure time series algorithms that generate accurate point forecasts, and two of these also produce forecast coverage probabilities which are satisfactorily close to the nominal rates. For annual data we find that Naïve forecasts are hard to beat.     

Forecasting sales response for multiple time horizons and temporally aggregated data: A comparison of constant and stochastic coefficient models

Past research on time-varying sales-response models emphasized the application of different estimation techniques in examining variation in advertising effectiveness over time. This study focuses on comparing sales forecasts using constant and stochastic coefficients sales-response models. Selected constant and stochastic coefficient models are applied to six sets of bimonthly and one set of annual advertising and sales data to assess forecasting accuracy for time horizons of various lengths. Results show improved forecasting accuracy for a first-order autoregressive stochastic coefficient model, particularly in short-run forecasting applications.     

Forecasting tourist arrivals using time-varying parameter structural time series models

Empirical evidence has shown that seasonal patterns of tourism demand and the effects of various influencing factors on this demand tend to change over time. To forecast future tourism demand accurately requires appropriate modelling of these changes. Based on the structural time series model (STSM) and the time-varying parameter (TVP) regression approach, this study develops the causal STSM further by introducing TVP estimation of the explanatory variable coefficients, and therefore combines the merits of the STSM and TVP models. This new model, the TVP-STSM, is employed for modelling and forecasting quarterly tourist arrivals to Hong Kong from four key source markets: China, South Korea, the UK and the USA. The empirical results show that the TVP-STSM outperforms all seven competitors, including the basic and causal STSMs and the TVP model for one- to four-quarter-ahead ex post forecasts and one-quarter-ahead ex ante forecasts.     

Reconciled boosted models for GEFCom2017 hierarchical probabilistic load forecasting

When forecasting time series in a hierarchical configuration, it is necessary to ensure that the forecasts reconcile at all levels. The 2017 Global Energy Forecasting Competition (GEFCom2017) focused on addressing this topic. Quantile forecasts for eight zones and two aggregated zones in New England were required for every hour of a future month. This paper presents a new methodology for forecasting quantiles in a hierarchy which outperforms a commonly-used benchmark model. A simulation-based approach was used to generate demand forecasts. Adjustments were made to each of the demand simulations to ensure that all zonal forecasts reconciled appropriately, and a weighted reconciliation approach was implemented to ensure that the bottom-level zonal forecasts summed correctly to the aggregated zonal forecasts. We show that reconciling in this manner improves the forecast accuracy. A discussion of the results and modelling performances is presented, and brief reviews of hierarchical time series forecasting and gradient boosting are also included.     

A copula-based time series model for global horizontal irradiation

The increasing importance of solar power for electricity generation leads to increasing demand for probabilistic forecasting of local and aggregated photovoltaic (PV) yields. Based on publicly available irradiation data, this paper uses an indirect modeling approach for hourly medium to long-term local PV yields. We suggest a time series model for global horizontal irradiation that allows for multivariate probabilistic forecasts for arbitrary time horizons. It features several important stylized facts. Sharp time-dependent lower and upper bounds of global horizontal irradiations are estimated. The parameters of the beta distributed marginals of the transformed data are allowed to be time-dependent. A copula-based time series model is introduced for the hourly and daily dependence structure based on simple vine copulas with so-called tail dependence. Evaluation methods based on scoring rules are used to compare the model’s power for multivariate probabilistic forecasting with other models used in the literature showing that our model outperforms other models in many respects.     

Improving forecast stability using deep learning

In this paper, we define forecast (in)stability in terms of the variability in forecasts for a specific time period caused by updating the forecast for this time period when new observations become available, i.e., as time passes. We propose an extension to the state-of-the-art N-BEATS deep learning architecture for the univariate time series point forecasting problem. The extension allows us to optimize forecasts from both a traditional forecast accuracy perspective as well as a forecast stability perspective. We show that the proposed extension results in forecasts that are more stable without leading to a deterioration in forecast accuracy for the M3 and M4 data sets. Moreover, our experimental study shows that it is possible to improve both forecast accuracy and stability compared to the original N-BEATS architecture, indicating that including a forecast instability component in the loss function can be used as regularization mechanism.     

Principles and algorithms for forecasting groups of time series: Locality and globality

Global methods that fit a single forecasting method to all time series in a set have recently shown surprising accuracy, even when forecasting large groups of heterogeneous time series. We provide the following contributions that help understand the potential and applicability of global methods and how they relate to traditional local methods that fit a separate forecasting method to each series:

• •Global and local methods can produce the same forecasts without any assumptions about similarity of the series in the set.
• •The complexity of local methods grows with the size of the set while it remains constant for global methods. This result supports the recent evidence and provides principles for the design of new algorithms.
• •In an extensive empirical study, we show that purposely naïve algorithms derived from these principles show outstanding accuracy. In particular, global linear models provide competitive accuracy with far fewer parameters than the simplest of local methods.

     

A SHARP model of bid–ask spread forecasts

This paper proposes an accurate, parsimonious and fast-to-estimate forecasting model for integer-valued time series with long memory and seasonality. The modelling is achieved through an autoregressive Poisson process with a predictable stochastic intensity that is determined by two factors: a seasonal intraday pattern and a heterogeneous autoregressive component. We call the model SHARP, which is an acronym for seasonal heterogeneous autoregressive Poisson. We also present a mixed-data sampling extension of the model, which adopts the historical information flow more efficiently and provides the best (among all the models considered) forecasting performances, empirically, for the bid–ask spreads of NYSE equity stocks. We conclude by showing how bid–ask spread forecasts based on the SHARP model can be exploited in order to reduce the total cost incurred by a trader who is willing to buy or sell a given amount of an equity stock.     

Assessing the accuracy of electricity production forecasts in developing countries

This study assesses the accuracy of time series econometric methods for forecasting electricity production in developing countries. An analysis of the historical time series for 106 developing countries over the period 1960–2012 demonstrates that econometric forecasts are highly accurate for the majority of these countries. These forecasts have much smaller errors than the predictions of simple heuristic models, which assume that electricity production grows at an exogenous rate or is proportional to the real GDP growth. However, the quality of the forecasts diminishes for the countries and regions, where rapid economic and structural transformation makes it difficult to establish stable historical production trends.     

A comprehensive evaluation of macroeconomic forecasting methods

We employ datasets for seven developed economies and consider four classes of multivariate forecasting models in order to extend and enhance the empirical evidence in the macroeconomic forecasting literature. The evaluation considers forecasting horizons of between one quarter and two years ahead. We find that the structural model, a medium-sized DSGE model, provides accurate long-horizon US and UK inflation forecasts. We strike a balance between being comprehensive and producing clear messages by applying meta-analysis regressions to 2,976 relative accuracy comparisons that vary with the forecasting horizon, country, model class and specification, number of predictors, and evaluation period. For point and density forecasting of GDP growth and inflation, we find that models with large numbers of predictors do not outperform models with 13–14 hand-picked predictors. Factor-augmented models and equal-weighted combinations of single-predictor mixed-data sampling regressions are a better choice for dealing with large numbers of predictors than Bayesian VARs.     

Forecasting with vector autoregressive (VAR) models subject to business cycle restrictions

In the last decade VAR models have become a widely-used tool for forecasting macroeconomic time series. To improve the out-of-sample forecasting accuracy of these models, Bayesian random-walk prior restrictions are often imposed on VAR model parameters. This paper focuses on whether placing an alternative type of restriction on the parameters of unrestricted VAR models improves the out-of-sample forecasting performance of these models. The type of restriction analyzed here is based on the business cycle characteristics of U.S. macroeconomic data, and in particular, requires that the dynamic behavior of the restricted VAR model mimic the business cycle characteristics of historical data. The question posed in this paper is: would a VAR model, estimated subject to the restriction that the cyclical characteristics of simulated data from the model “match up” with the business cycle characteristics of U.S. data, generate more accurate out-of-sample forecasts than unrestricted or Bayesian VAR models?     

When was the U.S. housing downturn predictable? A comparison of univariate forecasting methods

This paper uses three classes of univariate time series techniques (ARIMA type models, switching regression models, and state-space/structural time series models) to forecast, on an ex post basis, the downturn in U.S. housing prices starting around 2006. The performance of the techniques is compared within each class and across classes by out-of-sample forecasts for a number of different forecast points prior to and during the downturn. Most forecasting models are able to predict a downturn in future home prices by mid 2006. Some state-space models can predict an impending downturn as early as June 2005. State-space/structural time series models tend to produce the most accurate forecasts, although they are not necessarily the models with the best in-sample fit.     

FACTOR MODELS AND TIME‐VARYING PARAMETER FRAMEWORK FOR FORECASTING EXCHANGE RATES AND INFLATION: A SURVEY

A survey of models used for forecasting exchange rates and inflation reveals that the factor‐based and time‐varying parameter or state space models generate superior forecasts relative to all other models. This survey also finds that models based on Taylor rule and portfolio balance theory have moderate predictive power for forecasting exchange rates. The evidence on the use of Bayesian Model Averaging approach in forecasting exchange rates reveals limited predictive power, but strong support for forecasting inflation. Overall, the evidence overwhelmingly points to the context of the forecasts, relevance of the historical data, data transformation, choice of the benchmark, selected time horizons, sample period and forecast evaluation methods as the crucial elements in selecting forecasting models for exchange rate and inflation.     

Modelling and Forecasting Fiscal Variables for the Euro Area*

In this paper, we assess the possibility of producing unbiased forecasts for fiscal variables in the Euro area by comparing a set of procedures that rely on different information sets and econometric techniques. In particular, we consider autoregressive moving average models, Vector autoregressions, small‐scale semistructural models at the national and Euro area level, institutional forecasts (Organization for Economic Co‐operation and Development), and pooling. Our small‐scale models are characterized by the joint modelling of fiscal and monetary policy using simple rules, combined with equations for the evolution of all the relevant fundamentals for the Maastricht Treaty and the Stability and Growth Pact. We rank models on the basis of their forecasting performance using the mean square and mean absolute error criteria at different horizons. Overall, simple time‐series methods and pooling work well and are able to deliver unbiased forecasts, or slightly upward‐biased forecast for the debt–GDP dynamics. This result is mostly due to the short sample available, the robustness of simple methods to structural breaks, and to the difficulty of modelling the joint behaviour of several variables in a period of substantial institutional and economic changes. A bootstrap experiment highlights that, even when the data are generated using the estimated small‐scale multi‐country model, simple time‐series models can produce more accurate forecasts, because of their parsimonious specification.     

Seasonal exponential smoothing with damped trends : An application for production planning

This paper reviews a spreadsheet-based forecasting approach which a process industry manufacturer developed and implemented to link annual corporate forecasts with its manufacturing/distribution operations. First, we consider how this forecasting system supports overall production planning and why it must be compatible with corporate forecasts. We then review the results of substantial testing of variations on the Winters three-parameter exponential smoothing model on 28 actual product family time series. In particular, we evaluate whether the use of damping parameters improves forecast accuracy. The paper concludes that a Winters four-parameter model (i.e. the standard Winters three-parameter model augmented by a fourth parameter to damp the trend) provides the most accurate forecasts of the models evaluated. Our application confirms the fact that there are situations where the use of damped trend parameters in short-run exponential smoothing based forecasting models is beneficial.     

Comparing probabilistic forecasts of the daily minimum and maximum temperature

Understanding changes in the frequency, severity, and seasonality of daily temperature extremes is important for public policy decisions regarding heat waves and cold snaps. A heat wave is sometimes defined in terms of both the daily minimum and maximum temperature, which necessitates the generation of forecasts of their joint distribution. In this paper, we develop time series models with the aim of providing insight and producing forecasts of the joint distribution that can challenge the accuracy of forecasts based on ensemble predictions from a numerical weather prediction model. We use ensemble model output statistics to recalibrate the raw ensemble predictions for the marginal distributions, with ensemble copula coupling used to capture the dependency between the marginal distributions. In terms of time series modelling, we consider a bivariate VARMA-MGARCH model. We use daily Spanish data recorded over a 65-year period, and find that, for the 5-year out-of-sample period, the recalibrated ensemble predictions outperform the time series models in terms of forecast accuracy.     

The performance of the global bottom-up approach in the M5 accuracy competition: A robustness check

The M5 accuracy competition has presented a large-scale hierarchical forecasting problem in a realistic grocery retail setting in order to evaluate an extended range of forecasting methods, particularly those adopting machine learning. The top ranking solutions adopted a global bottom-up approach, by which is meant using global forecasting methods to generate bottom level forecasts in the hierarchy and then using a bottom-up strategy to obtain coherent forecasts for aggregate levels. However, whether the observed superior performance of the global bottom-up approach is robust over various test periods or only an accidental result, is an important question for retail forecasting researchers and practitioners. We conduct experiments to explore the robustness of the global bottom-up approach, and make comments on the efforts made by the top-ranking teams to improve the core approach. We find that the top-ranking global bottom-up approaches lack robustness across time periods in the M5 data. This inconsistent performance makes the M5 final rankings somewhat of a lottery. In future forecasting competitions, we suggest the use of multiple rolling test sets to evaluate the forecasting performance in order to reward robustly performing forecasting methods, a much needed characteristic in any application.