LoMEF: A framework to produce local explanations for global model time series forecasts

Global forecasting models (GFMs) that are trained across a set of multiple time series have shown superior results in many forecasting competitions and real-world applications compared with univariate forecasting approaches. One aspect of the popularity of statistical forecasting models such as ETS and ARIMA is their relative simplicity and interpretability (in terms of relevant lags, trend, seasonality, and other attributes), while GFMs typically lack interpretability, especially relating to particular time series. This reduces the trust and confidence of stakeholders when making decisions based on the forecasts without being able to understand the predictions. To mitigate this problem, we propose a novel local model-agnostic interpretability approach to explain the forecasts from GFMs. We train simpler univariate surrogate models that are considered interpretable (e.g., ETS) on the predictions of the GFM on samples within a neighbourhood that we obtain through bootstrapping, or straightforwardly as the one-step-ahead global black-box model forecasts of the time series which needs to be explained. After, we evaluate the explanations for the forecasts of the global models in both qualitative and quantitative aspects such as accuracy, fidelity, stability, and comprehensibility, and are able to show the benefits of our approach.     

Expectation-based scan statistics for monitoring spatial time series data

We consider the simultaneous monitoring of a large number of spatially localized time series in order to detect emerging spatial patterns. For example, in disease surveillance, we detect emerging outbreaks by monitoring electronically available public health data, e.g. aggregate daily counts of Emergency Department visits. We propose a two-step approach based on the expectation-based scan statistic: we first compute the expected count for each recent day for each spatial location, then find spatial regions (groups of nearby locations) where the recent counts are significantly higher than expected. By aggregating information across multiple time series rather than monitoring each series separately, we can improve the timeliness, accuracy, and spatial resolution of detection. We evaluate several variants of the expectation-based scan statistic on the disease surveillance task (using synthetic outbreaks injected into real-world hospital Emergency Department data), and draw conclusions about which models and methods are most appropriate for which surveillance tasks.     

Principles and algorithms for forecasting groups of time series: Locality and globality

Global methods that fit a single forecasting method to all time series in a set have recently shown surprising accuracy, even when forecasting large groups of heterogeneous time series. We provide the following contributions that help understand the potential and applicability of global methods and how they relate to traditional local methods that fit a separate forecasting method to each series:

• •Global and local methods can produce the same forecasts without any assumptions about similarity of the series in the set.
• •The complexity of local methods grows with the size of the set while it remains constant for global methods. This result supports the recent evidence and provides principles for the design of new algorithms.
• •In an extensive empirical study, we show that purposely naïve algorithms derived from these principles show outstanding accuracy. In particular, global linear models provide competitive accuracy with far fewer parameters than the simplest of local methods.

     

Forecasting residential burglary

Following the work of Dhiri et al. [Modelling and predicting property crime trends. Home Office Research Study 198 (1999). London: HMSO] at the Home Office predicting recorded burglary and theft for England and Wales to the year 2001, econometric and time series models were constructed for predicting recorded residential burglary to the same date. A comparison between the Home Office econometric predictions and the less alarming econometric predictions made in this paper identified the differences as stemming from the particular set of variables used in the models. However, the Home Office and one of our econometric models adopted an error correction form which appeared to be the main reason why these models predicted increases in burglary. To identify the role of error correction in these models, time series models were built for the purpose of comparison, all of which predicted substantially lower numbers of residential burglaries. The years 1998–2001 appeared to offer an opportunity to test the utility of error correction models in the analysis of criminal behaviour. Subsequent to the forecasting exercise carried out in 1999, recorded outcomes have materialised, which point to the superiority of time series models compared to error correction models for the short-run forecasting of property crime. This result calls into question the concept of a long-run equilibrium relationship for crime.     

FFORMPP: Feature-based forecast model performance prediction

This paper introduces a novel meta-learning algorithm for time series forecast model performance prediction. We model the forecast error as a function of time series features calculated from historical time series with an efficient Bayesian multivariate surface regression approach. The minimum predicted forecast error is then used to identify an individual model or a combination of models to produce the final forecasts. It is well known that the performance of most meta-learning models depends on the representativeness of the reference dataset used for training. In such circumstances, we augment the reference dataset with a feature-based time series simulation approach, namely GRATIS, to generate a rich and representative time series collection. The proposed framework is tested using the M4 competition data and is compared against commonly used forecasting approaches. Our approach provides comparable performance to other model selection and combination approaches but at a lower computational cost and a higher degree of interpretability, which is important for supporting decisions. We also provide useful insights regarding which forecasting models are expected to work better for particular types of time series, the intrinsic mechanisms of the meta-learners, and how the forecasting performance is affected by various factors.     

Endemic-epidemic models with discrete-time serial interval distributions for infectious disease prediction

Multivariate count time series models are an important tool for analyzing and predicting the spread of infectious disease. We consider the endemic-epidemic framework, a class of autoregressive models for infectious disease surveillance counts, and replace the default autoregression on counts from the previous time period with more flexible weighting schemes inspired by discrete-time serial interval distributions. We employ three different parametric formulations, each with an additional unknown weighting parameter estimated via a profile likelihood approach, and compare them to an unrestricted nonparametric approach. The new methods are illustrated in a univariate analysis of dengue fever incidence in San Juan, Puerto Rico, and a spatiotemporal study of viral gastroenteritis in the 12 districts of Berlin. We assess the predictive performance of the suggested models and several reference models at various forecast horizons. In both applications, the performance of the endemic-epidemic models is considerably improved by the proposed weighting schemes.     

Evaluating factor forecasts for the UK: The role of asset prices

This paper applies a large data set, consisting of 167 monthly time series for the UK, both economic and financial, to simulate out-of-sample predictions of industrial production, inflation, 3-month Treasury Bills, and other variables. Fifteen dynamic factor models that allow forecasting based on large panels of time series are considered. The performances of these factor models are then compared to the following competing models: a simple univariate autoregressive, a vector autoregressive, a leading indicator, and a Phillips curve models. The results show that the best dynamic factor models outperform the competing models in forecasting at 6-, 12-, and 24-month horizons. Thus, the financial markets may have predictive power for the economic activity. This can be a useful tool for central banks and financial institutions, which may use the factor models to construct leading indicators of the economic conditions. In addition, researchers can see a strategic application of factor models.     

U-Convolutional model for spatio-temporal wind speed forecasting

The increasing penetration of intermittent renewable energy in power systems brings operational challenges. One way of supporting them is by enhancing the predictability of renewables through accurate forecasting. Convolutional Neural Networks (Convnets) provide a successful technique for processing space-structured multi-dimensional data. In our work, we propose the U-Convolutional model to predict hourly wind speeds for a single location using spatio-temporal data with multiple explanatory variables as an input. The U-Convolutional model is composed of a U-Net part, which synthesizes input information, and a Convnet part, which maps the synthesized data into a single-site wind prediction. We compare our approach with advanced Convnets, a fully connected neural network, and univariate models. We use time series from the Climate Forecast System Reanalysis as datasets and select temperature and u- and v-components of wind as explanatory variables. The proposed models are evaluated at multiple locations (totaling 181 target series) and multiple forecasting horizons. The results indicate that our proposal is promising for spatio-temporal wind speed prediction, with results that show competitive performance on both time horizons for all datasets.     

Forecasting the NN5 time series with hybrid models

We propose a simple way of predicting time series with recurring seasonal periods. Missing values of the time series are estimated and interpolated in a preprocessing step. We combine several forecasting methods by taking the weighted mean of forecasts that were generated with time-domain models which were validated on left-out parts of the time series. The hybrid model is a combination of a neural network ensemble, an ensemble of nearest trajectory models and a model for the 7-day cycle. We apply this approach to the NN5 time series competition data set.     

Kaggle forecasting competitions: An overlooked learning opportunity

We review the results of six forecasting competitions based on the online data science platform Kaggle, which have been largely overlooked by the forecasting community. In contrast to the M competitions, the competitions reviewed in this study feature daily and weekly time series with exogenous variables, business hierarchy information, or both. Furthermore, the Kaggle data sets all exhibit higher entropy than the M3 and M4 competitions, and they are intermittent.In this review, we confirm the conclusion of the M4 competition that ensemble models using cross-learning tend to outperform local time series models and that gradient boosted decision trees and neural networks are strong forecast methods. Moreover, we present insights regarding the use of external information and validation strategies, and discuss the impacts of data characteristics on the choice of statistics or machine learning methods. Based on these insights, we construct nine ex-ante hypotheses for the outcome of the M5 competition to allow empirical validation of our findings.     

FFORMA: Feature-based forecast model averaging

We propose an automated method for obtaining weighted forecast combinations using time series features. The proposed approach involves two phases. First, we use a collection of time series to train a meta-model for assigning weights to various possible forecasting methods with the goal of minimizing the average forecasting loss obtained from a weighted forecast combination. The inputs to the meta-model are features that are extracted from each series. Then, in the second phase, we forecast new series using a weighted forecast combination, where the weights are obtained from our previously trained meta-model. Our method outperforms a simple forecast combination, as well as all of the most popular individual methods in the time series forecasting literature. The approach achieved second position in the M4 competition.     

A note on the Robust Trend and ARARMA methodologies used in the M3 Competition

This brief note describes two of the forecasting methods used in the M3 Competition, Robust Trend and ARARMA. The origins of these methods are very different. Robust Trend was introduced to model the special features of some telecommunications time series. It was subsequently found to be competitive with Holt’s linear model for the more varied set of time series used in the M1 Competition. The ARARMA methodology was proposed by Parzen as a general time series modelling procedure, and can be thought of as an alternative to the ARIMA methodology of Box and Jenkins. This method was used in the M1 Competition and achieved the lowest mean absolute percentage error for longer forecasting horizons. These methods will be described in more detail and some comments on their use in the M3 Competition conclude this note.     

Online hierarchical forecasting for power consumption data

This paper proposes a three-step approach to forecasting time series of electricity consumption at different levels of household aggregation. These series are linked by hierarchical constraints—global consumption is the sum of regional consumption, for example. First, benchmark forecasts are generated for all series using generalized additive models. Second, for each series, the aggregation algorithm ML-Poly, introduced by Gaillard, Stoltz, and van Erven in 2014, finds an optimal linear combination of the benchmarks. Finally, the forecasts are projected onto a coherent subspace to ensure that the final forecasts satisfy the hierarchical constraints. By minimizing a regret criterion, we show that the aggregation and projection steps improve the root mean square error of the forecasts. Our approach is tested on household electricity consumption data; experimental results suggest that successive aggregation and projection steps improve the benchmark forecasts at different levels of household aggregation.     

Evaluating forecasts of political conflict dynamics

There is considerable interest today in the forecasting of conflict dynamics. Commonly, the root mean square error and other point metrics are used to evaluate the forecasts from such models. However, conflict processes are non-linear, so these point metrics often do not produce adequate evaluations of the calibration and sharpness of the forecast models. Forecast density evaluation improves the model evaluation. We review tools for density evaluation, including continuous rank probability scores, verification rank histograms, and sharpness plots. The usefulness of these tools for evaluating conflict forecasting models is explained. We illustrate this, first, in a comparison of several time series models’ forecasts of simulated data from a Markov-switching process, and second, in a comparison of several models’ abilities to forecast conflict dynamics in the Cross Straits. These applications show the pitfalls of relying on point metrics alone for evaluating the quality of conflict forecasting models. As in other fields, it is more useful to employ a suite of tools. A non-linear vector autoregressive model emerges as the model which is best able to forecast conflict dynamics between China and Taiwan.     

Estimation and model adequacy checking for multivariate seasonal autoregressive time series models with periodically varying parameters

We introduce a class of multivariate seasonal time series models with periodically varying parameters, abbreviated by the acronym SPVAR. The model is suitable for multivariate data, and combines a periodic autoregressive structure and a multiplicative seasonal time series model. The stationarity conditions (in the periodic sense) and the theoretical autocovariance functions of SPVAR stochastic processes are derived. Estimation and checking stages are considered. The asymptotic normal distribution of the least squares estimators of the model parameters is established, and the asymptotic distributions of the residual autocovariance and autocorrelation matrices in the class of SPVAR time series models are obtained. In order to check model adequacy, portmanteau test statistics are considered and their asymptotic distributions are studied. A simulation study is briefly discussed to investigate the finite-sample properties of the proposed test statistics. The methodology is illustrated with a bivariate quarterly data set on travelers entering in to Canada.     

When was the U.S. housing downturn predictable? A comparison of univariate forecasting methods

This paper uses three classes of univariate time series techniques (ARIMA type models, switching regression models, and state-space/structural time series models) to forecast, on an ex post basis, the downturn in U.S. housing prices starting around 2006. The performance of the techniques is compared within each class and across classes by out-of-sample forecasts for a number of different forecast points prior to and during the downturn. Most forecasting models are able to predict a downturn in future home prices by mid 2006. Some state-space models can predict an impending downturn as early as June 2005. State-space/structural time series models tend to produce the most accurate forecasts, although they are not necessarily the models with the best in-sample fit.     

Investigating the accuracy of cross-learning time series forecasting methods

The M4 competition identified innovative forecasting methods, advancing the theory and practice of forecasting. One of the most promising innovations of M4 was the utilization of cross-learning approaches that allow models to learn from multiple series how to accurately predict individual ones. In this paper, we investigate the potential of cross-learning by developing various neural network models that adopt such an approach, and we compare their accuracy to that of traditional models that are trained in a series-by-series fashion. Our empirical evaluation, which is based on the M4 monthly data, confirms that cross-learning is a promising alternative to traditional forecasting, at least when appropriate strategies for extracting information from large, diverse time series data sets are considered. Ways of combining traditional with cross-learning methods are also examined in order to initiate further research in the field.     

A nonlinear long memory model, with an application to US unemployment

Two important empirical features of US unemployment are that shocks to the series seem rather persistent and that it seems to rise faster during recessions than that it falls during expansions. To jointly capture these features of long memory and nonlinearity, we put forward a new time series model and evaluate its empirical performance. We find that the model describes the data rather well and that it outperforms related competitive models on various measures of fit.     

Shrinkage estimators of time series seasonal factors and their effect on forecasting accuracy

This paper shows that forecasting accuracy can be improved through better estimation of seasonal factors under conditions for which relatively simple methods are preferred, such as relatively few historical data, noisy data, and/or a large number of series to be forecasted. In such situations, the preferred method of seasonal adjustment is often ratio-to-moving-averages (classical) decomposition. This paper proposes two shrinkage estimators to improve the accuracy of classical decomposition seasonal factors. In a simulation study, both of the proposed estimators provided consistently greater accuracy than classical decomposition, with the improvement sometimes being dramatic. The performances of the two estimators depended on characteristics of the series, and guidelines were developed for choosing one of them under a given set of conditions. For a set of monthly, M-competition series, greater forecasting accuracy was achieved when either of the proposed methods was used for seasonal adjustment rather than classical decomposition, and the greatest accuracy was achieved by following the guidelines for choosing a method.     

Small Area Estimation-New Developments and Directions

The purpose of this paper is to provide a critical review of the main advances in small area estimation (SAE) methods in recent years. We also discuss some of the earlier developments, which serve as a necessary background for the new studies. The review focuses on model dependent methods with special emphasis on point prediction of the target area quantities, and mean square error assessments. The new models considered are models used for discrete measurements, time series models and models that arise under informative sampling. The possible gains from modeling the correlations among small area random effects used to represent the unexplained variation of the small area target quantities are examined. For review and appraisal of the earlier methods used for SAE, see Ghosh & Rao (1994).