Forecasting correlated time series with exponential smoothing models

This paper presents the Bayesian analysis of a general multivariate exponential smoothing model that allows us to forecast time series jointly, subject to correlated random disturbances. The general multivariate model, which can be formulated as a seemingly unrelated regression model, includes the previously studied homogeneous multivariate Holt-Winters’ model as a special case when all of the univariate series share a common structure. MCMC simulation techniques are required in order to approach the non-analytically tractable posterior distribution of the model parameters. The predictive distribution is then estimated using Monte Carlo integration. A Bayesian model selection criterion is introduced into the forecasting scheme for selecting the most adequate multivariate model for describing the behaviour of the time series under study. The forecasting performance of this procedure is tested using some real examples.     

A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting

This paper presents the winning submission of the M4 forecasting competition. The submission utilizes a dynamic computational graph neural network system that enables a standard exponential smoothing model to be mixed with advanced long short term memory networks into a common framework. The result is a hybrid and hierarchical forecasting method.     

Improving time series forecasting: An approach combining bootstrap aggregation,clusters and exponential smoothing

Some recent papers have demonstrated that combining bagging (bootstrap aggregating) with exponential smoothing methods can produce highly accurate forecasts and improve the forecast accuracy relative to traditional methods. We therefore propose a new approach that combines the bagging, exponential smoothing and clustering methods. The existing methods use bagging to generate and aggregate groups of forecasts in order to reduce the variance. However, none of them consider the effect of covariance among the group of forecasts, even though it could have a dramatic impact on the variance of the group, and therefore on the forecast accuracy. The proposed approach, referred to here as Bagged.Cluster.ETS, aims to reduce the covariance effect by using partitioning around medoids (PAM) to produce clusters of similar forecasts, then selecting several forecasts from each cluster to create a group with a reduced variance. This approach was tested on various different time series sets from the M3 and CIF 2016 competitions. The empirical results have shown a substantial reduction in the forecast error, considering sMAPE and MASE.     

Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition

In this work we introduce the forecasting model with which we participated in the NN5 forecasting competition (the forecasting of 111 time series representing daily cash withdrawal amounts at ATM machines). The main idea of this model is to utilize the concept of forecast combination, which has proven to be an effective methodology in the forecasting literature. In the proposed system we attempted to follow a principled approach, and make use of some of the guidelines and concepts that are known in the forecasting literature to lead to superior performance. For example, we considered various previous comparison studies and time series competitions as guidance in determining which individual forecasting models to test (for possible inclusion in the forecast combination system). The final model ended up consisting of neural networks, Gaussian process regression, and linear models, combined by simple average. We also paid extra attention to the seasonality aspect, decomposing the seasonality into weekly (which is the strongest one), day of the month, and month of the year seasonality.     

Time series modeling of histogram-valued data: The daily histogram time series of S&P500 intradaily returns

Histogram time series (HTS) and interval time series (ITS) are examples of symbolic data sets. Though there have been methodological developments in a cross-sectional environment, they have been scarce in a time series setting. Arroyo, González-Rivera, and Maté (2011) analyze various forecasting methods for HTS and ITS, adapting smoothing filters and nonparametric algorithms such as the k-NN. Though these methods are very flexible, they may not be the true underlying data generating process (DGP). We present the first step in the search for a DGP by focusing on the autocorrelation functions (ACFs) of HTS and ITS. We analyze the ACF of the daily histogram of 5-minute intradaily returns to the S&P500 index in 2007 and 2008. There are clusters of high/low activity that generate a strong, positive, and persistent autocorrelation, pointing towards some autoregressive process for HTS. Though smoothing and k-NN may not be the true DGPs, we find that they are very good approximations because they are able to capture almost all of the original autocorrelation. However, there seems to be some structure left in the data that will require new modelling techniques. As a byproduct, we also analyze the [90,100%] quantile interval. By using all of the information contained in the histogram, we find that there are advantages in the estimation and prediction of a specific interval.     

Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination

In this paper, we examine the forecast accuracy of linear autoregressive, smooth transition autoregressive (STAR), and neural network (NN) time series models for 47 monthly macroeconomic variables of the G7 economies. Unlike previous studies that typically consider multiple but fixed model specifications, we use a single but dynamic specification for each model class. The point forecast results indicate that the STAR model generally outperforms linear autoregressive models. It also improves upon several fixed STAR models, demonstrating that careful specification of nonlinear time series models is of crucial importance. The results for neural network models are mixed in the sense that at long forecast horizons, an NN model obtained using Bayesian regularization produces more accurate forecasts than a corresponding model specified using the specific-to-general approach. Reasons for this outcome are discussed.     

Blending gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series

In this study, we addressed the problem of point and probabilistic forecasting by describing a blending methodology for machine learning models from the gradient boosted trees and neural networks families. These principles were successfully applied in the recent M5 Competition in both the Accuracy and Uncertainty tracks. The key points of our methodology are: (a) transforming the task into regression on sales for a single day; (b) information-rich feature engineering; (c) creating a diverse set of state-of-the-art machine learning models; and (d) carefully constructing validation sets for model tuning. We show that the diversity of the machine learning models and careful selection of validation examples are most important for the effectiveness of our approach. Forecasting data have an inherent hierarchical structure (12 levels) but none of our proposed solutions exploited the hierarchical scheme. Using the proposed methodology, we ranked within the gold medal range in the Accuracy track and within the prizes in the Uncertainty track. Inference code with pre-trained models are available on GitHub.¹     

Machine learning advances for time series forecasting

In this paper, we survey the most recent advances in supervised machine learning (ML) and high-dimensional models for time-series forecasting. We consider both linear and nonlinear alternatives. Among the linear methods, we pay special attention to penalized regressions and ensemble of models. The nonlinear methods considered in the paper include shallow and deep neural networks, in their feedforward and recurrent versions, and tree-based methods, such as random forests and boosted trees. We also consider ensemble and hybrid models by combining ingredients from different alternatives. Tests for superior predictive ability are briefly reviewed. Finally, we discuss application of ML in economics and finance and provide an illustration with high-frequency financial data.     

An accurate and fully-automated ensemble model for weekly time series forecasting

Many businesses and industries require accurate forecasts for weekly time series nowadays. However, the forecasting literature does not currently provide easy-to-use, automatic, reproducible and accurate approaches dedicated to this task. We propose a forecasting method in this domain to fill this gap, leveraging state-of-the-art forecasting techniques, such as forecast combination, meta-learning, and global modelling. We consider different meta-learning architectures, algorithms, and base model pools. Based on all considered model variants, we propose to use a stacking approach with lasso regression which optimally combines the forecasts of four base models: a global Recurrent Neural Network (RNN) model, Theta, Trigonometric Box–Cox ARMA Trend Seasonal (TBATS), and Dynamic Harmonic Regression ARIMA (DHR-ARIMA), as it shows the overall best performance across seven experimental weekly datasets on four evaluation metrics. Our proposed method also consistently outperforms a set of benchmarks and state-of-the-art weekly forecasting models by a considerable margin with statistical significance. Our method can produce the most accurate forecasts, in terms of mean sMAPE, for the M4 weekly dataset among all benchmarks and all original competition participants.     

Automatic neural network modeling for univariate time series

Artificial neural networks (ANNs) are an information processing paradigm inspired by the way the brain processes information. Using neural networks requires the investigator to make decisions concerning the architecture or structure used. ANNs are known to be universal function approximators and are capable of exploiting nonlinear relationships between variables. This method, called Automated ANNs, is an attempt to develop an automatic procedure for selecting the architecture of an artificial neural network for forecasting purposes. It was entered into the M-3 Time Series Competition. Results show that ANNs compete well with the other methods investigated, but may produce poor results if used under certain conditions.     

The challenges of pre-launch forecasting of adoption time series for new durable products

The successful introduction of new durable products plays an important part in helping companies to stay ahead of their competitors. Decisions relating to these products can be improved by the availability of reliable pre-launch forecasts of their adoption time series. However, producing such forecasts is a difficult, complex and challenging task, mainly because of the non-availability of past time series data relating to the product, and the multiple factors that can affect adoptions, such as customer heterogeneity, macroeconomic conditions following the product launch, and technological developments which may lead to the product’s premature obsolescence. This paper provides a critical review of the literature to examine what it can tell us about the relative effectiveness of three fundamental approaches to filling the data void : (i) management judgment, (ii) the analysis of judgments by potential customers, and (iii) formal models of the diffusion process. It then shows that the task of producing pre-launch time series forecasts of adoption levels involves a set of sub-tasks, which all involve either quantitative estimation or choice, and argues that the different natures of these tasks mean that the forecasts are unlikely to be accurate if a single method is employed. Nevertheless, formal models should be at the core of the forecasting process, rather than unstructured judgment. Gaps in the literature are identified, and the paper concludes by suggesting a research agenda so as to indicate where future research efforts might be employed most profitably.     

Bagged neural networks for forecasting Polish (low) inflation

Accurate inflation forecasts lie at the heart of effective monetary policy. This paper utilizes a thick modelling approach in order to investigate the quality of the out-of-sample short-term headline inflation forecasts generated by a combination of bagged single hidden-layer feed-forward artificial neural networks. The model’s accuracy rises during the period of consistently falling and persistently low inflation in the emerging economy of Poland, and it statistically outperforms some of the popular benchmarks more frequently, especially at longer horizons. However, dispensing with data preprocessing and bootstrap aggregation compromises its forecasting ability severely. Combining linear and non-linear univariate and multivariate approaches with diverse underlying model assumptions delivers further gains in predictive accuracy and statistically outperforms a panel of benchmarks in a number of cases. While the vague interpretability of the model poses a considerable hurdle for policy makers, its inclusion in the forecasting toolbox should increase the accuracy of the ensemble of models, especially in periods of structural change.     

Validation and forecasting accuracy in models of climate change

Forecasting researchers, with few exceptions, have ignored the current major forecasting controversy: global warming and the role of climate modelling in resolving this challenging topic. In this paper, we take a forecaster’s perspective in reviewing established principles for validating the atmospheric-ocean general circulation models (AOGCMs) used in most climate forecasting, and in particular by the Intergovernmental Panel on Climate Change (IPCC). Such models should reproduce the behaviours characterising key model outputs, such as global and regional temperature changes. We develop various time series models and compare them with forecasts based on one well-established AOGCM from the UK Hadley Centre. Time series models perform strongly, and structural deficiencies in the AOGCM forecasts are identified using encompassing tests. Regional forecasts from various GCMs had even more deficiencies. We conclude that combining standard time series methods with the structure of AOGCMs may result in a higher forecasting accuracy. The methodology described here has implications for improving AOGCMs and for the effectiveness of environmental control policies which are focussed on carbon dioxide emissions alone. Critically, the forecast accuracy in decadal prediction has important consequences for environmental planning, so its improvement through this multiple modelling approach should be a priority.     

Distributed ARIMA models for ultra-long time series

Providing forecasts for ultra-long time series plays a vital role in various activities, such as investment decisions, industrial production arrangements, and farm management. This paper develops a novel distributed forecasting framework to tackle the challenges of forecasting ultra-long time series using the industry-standard MapReduce framework. The proposed model combination approach retains the local time dependency. It utilizes a straightforward splitting across samples to facilitate distributed forecasting by combining the local estimators of time series models delivered from worker nodes and minimizing a global loss function. Instead of unrealistically assuming the data generating process (DGP) of an ultra-long time series stays invariant, we only make assumptions on the DGP of subseries spanning shorter time periods. We investigate the performance of the proposed approach with AutoRegressive Integrated Moving Average (ARIMA) models using the real data application as well as numerical simulations. Our approach improves forecasting accuracy and computational efficiency in point forecasts and prediction intervals, especially for longer forecast horizons, compared to directly fitting the whole data with ARIMA models. Moreover, we explore some potential factors that may affect the forecasting performance of our approach.     

A two-step approach for identifying seasonal autoregressive time series forecasting models

One of the most powerful and widely used methodologies for forecasting economic time series is the class of models known as seasonal autoregressive processes. In this article we present a new approach not only for identifying seasonal autoregressive models, but also the degree of differencing required to induce stationarity in the data. The identification method is iterative and consists in systematically fitting increasing order models to the data, and then verifying that the resulting residuals behave like white noise using a two stage autoregressive order determination criterion. Once the order of the process is determined the identified structure is tested to see if it can be simplified. The identification performance of this procedure is contrasted with other order selection procedures for models with ‘gaps.' We also illustrate the forecast performance of the identification method using monthly and quarterly economic data.     

COVID-19: Forecasting confirmed cases and deaths with a simple time series model

Forecasting the outcome of outbreaks as early and as accurately as possible is crucial for decision-making and policy implementations. A significant challenge faced by forecasters is that not all outbreaks and epidemics turn into pandemics, making the prediction of their severity difficult. At the same time, the decisions made to enforce lockdowns and other mitigating interventions versus their socioeconomic consequences are not only hard to make, but also highly uncertain. The majority of modeling approaches to outbreaks, epidemics, and pandemics take an epidemiological approach that considers biological and disease processes. In this paper, we accept the limitations of forecasting to predict the long-term trajectory of an outbreak, and instead, we propose a statistical, time series approach to modelling and predicting the short-term behavior of COVID-19. Our model assumes a multiplicative trend, aiming to capture the continuation of the two variables we predict (global confirmed cases and deaths) as well as their uncertainty. We present the timeline of producing and evaluating 10-day-ahead forecasts over a period of four months. Our simple model offers competitive forecast accuracy and estimates of uncertainty that are useful and practically relevant.     

A fast and scalable ensemble of global models with long memory and data partitioning for the M5 forecasting competition

This work presents key insights on the model development strategies used in our cross-learning-based retail demand forecast framework. The proposed framework outperforms state-of-the-art univariate models in the time series forecasting literature. It has achieved 17th position in the accuracy track of the M5 forecasting competition, which is among the top 1% of solutions.     

Graphical interaction models for multivariate time series1

In this paper we extend the concept of graphical models for multivariate data to multivariate time series. We define a partial correlation graph for time series and use the partial spectral coherence between two components given the remaining components to identify the edges of the graph. As an example we consider multivariate autoregressive processes. The method is applied to air pollution data. Received: June 1999     

Optimal transformations for categorical autoregressive time series

This paper describes a method for finding optimal transformations for analyzing time series by autoregressive models. 'Optimal' implies that the agreement between the autoregressive model and the transformed data is maximal. Such transformations help 1) to increase the model fit, and 2) to analyze categorical time series. The method uses an alternating least squares algorithm that consists of two main steps: estimation and transformation. Nominal, ordinal and numerical data can be analyzed. Some alternative applications of the general idea are highlighted: intervention analysis, smoothing categorical time series, predictable components, spatial modeling and cross-sectional multivariate analysis. Limitations, modeling issues and possible extensions are briefly indicated.