The second-order bias and mean squared error of estimators in time-series models

We develop analytical results on the second-order bias and mean squared error of estimators in time-series models. These results provide a unified approach to developing the properties of a large class of estimators in linear and nonlinear time-series models and they are valid for both normal and nonnormal samples of observations, and where the regressors are stochastic. The estimators included are the generalized method of moments, maximum likelihood, least squares, and other extremum estimators. Our general results are applied to four time-series models. We investigate the effects of nonnormality on the second-order bias results for two of these models, while for all four models, the second-order bias and mean squared error results are given under normality. Numerical results for some of these models are also presented.     

Consistent estimation of a general nonparametric regression function in time series

We propose an estimator of the conditional distribution of X_t|X_t−1,X_t−2,…, and the corresponding regression function , where the conditioning set is of infinite order. We establish consistency of our estimator under stationarity and ergodicity conditions plus a mild smoothness condition.     

Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition

In this work we introduce the forecasting model with which we participated in the NN5 forecasting competition (the forecasting of 111 time series representing daily cash withdrawal amounts at ATM machines). The main idea of this model is to utilize the concept of forecast combination, which has proven to be an effective methodology in the forecasting literature. In the proposed system we attempted to follow a principled approach, and make use of some of the guidelines and concepts that are known in the forecasting literature to lead to superior performance. For example, we considered various previous comparison studies and time series competitions as guidance in determining which individual forecasting models to test (for possible inclusion in the forecast combination system). The final model ended up consisting of neural networks, Gaussian process regression, and linear models, combined by simple average. We also paid extra attention to the seasonality aspect, decomposing the seasonality into weekly (which is the strongest one), day of the month, and month of the year seasonality.     

Shrinkage estimators of time series seasonal factors and their effect on forecasting accuracy

This paper shows that forecasting accuracy can be improved through better estimation of seasonal factors under conditions for which relatively simple methods are preferred, such as relatively few historical data, noisy data, and/or a large number of series to be forecasted. In such situations, the preferred method of seasonal adjustment is often ratio-to-moving-averages (classical) decomposition. This paper proposes two shrinkage estimators to improve the accuracy of classical decomposition seasonal factors. In a simulation study, both of the proposed estimators provided consistently greater accuracy than classical decomposition, with the improvement sometimes being dramatic. The performances of the two estimators depended on characteristics of the series, and guidelines were developed for choosing one of them under a given set of conditions. For a set of monthly, M-competition series, greater forecasting accuracy was achieved when either of the proposed methods was used for seasonal adjustment rather than classical decomposition, and the greatest accuracy was achieved by following the guidelines for choosing a method.     

A unified approach to validating univariate and multivariate conditional distribution models in time series

Modeling conditional distributions in time series has attracted increasing attention in economics and finance. We develop a new class of generalized Cramer–von Mises (GCM) specification tests for time series conditional distribution models using a novel approach, which embeds the empirical distribution function in a spectral framework. Our tests check a large number of lags and are therefore expected to be powerful against neglected dynamics at higher order lags, which is particularly useful for non-Markovian processes. Despite using a large number of lags, our tests do not suffer much from loss of a large number of degrees of freedom, because our approach naturally downweights higher order lags, which is consistent with the stylized fact that economic or financial markets are more affected by recent past events than by remote past events. Unlike the existing methods in the literature, the proposed GCM tests cover both univariate and multivariate conditional distribution models in a unified framework. They exploit the information in the joint conditional distribution of underlying economic processes. Moreover, a class of easy-to-interpret diagnostic procedures are supplemented to gauge possible sources of model misspecifications. Distinct from conventional CM and Kolmogorov–Smirnov (KS) tests, which are also based on the empirical distribution function, our GCM test statistics follow a convenient asymptotic $N(0,1)$ distribution and enjoy the appealing “nuisance parameter free” property that parameter estimation uncertainty has no impact on the asymptotic distribution of the test statistics. Simulation studies show that the tests provide reliable inference for sample sizes often encountered in economics and finance.     

Distributed ARIMA models for ultra-long time series

Providing forecasts for ultra-long time series plays a vital role in various activities, such as investment decisions, industrial production arrangements, and farm management. This paper develops a novel distributed forecasting framework to tackle the challenges of forecasting ultra-long time series using the industry-standard MapReduce framework. The proposed model combination approach retains the local time dependency. It utilizes a straightforward splitting across samples to facilitate distributed forecasting by combining the local estimators of time series models delivered from worker nodes and minimizing a global loss function. Instead of unrealistically assuming the data generating process (DGP) of an ultra-long time series stays invariant, we only make assumptions on the DGP of subseries spanning shorter time periods. We investigate the performance of the proposed approach with AutoRegressive Integrated Moving Average (ARIMA) models using the real data application as well as numerical simulations. Our approach improves forecasting accuracy and computational efficiency in point forecasts and prediction intervals, especially for longer forecast horizons, compared to directly fitting the whole data with ARIMA models. Moreover, we explore some potential factors that may affect the forecasting performance of our approach.     

Expectation-based scan statistics for monitoring spatial time series data

We consider the simultaneous monitoring of a large number of spatially localized time series in order to detect emerging spatial patterns. For example, in disease surveillance, we detect emerging outbreaks by monitoring electronically available public health data, e.g. aggregate daily counts of Emergency Department visits. We propose a two-step approach based on the expectation-based scan statistic: we first compute the expected count for each recent day for each spatial location, then find spatial regions (groups of nearby locations) where the recent counts are significantly higher than expected. By aggregating information across multiple time series rather than monitoring each series separately, we can improve the timeliness, accuracy, and spatial resolution of detection. We evaluate several variants of the expectation-based scan statistic on the disease surveillance task (using synthetic outbreaks injected into real-world hospital Emergency Department data), and draw conclusions about which models and methods are most appropriate for which surveillance tasks.     

Graphical interaction models for multivariate time series1

In this paper we extend the concept of graphical models for multivariate data to multivariate time series. We define a partial correlation graph for time series and use the partial spectral coherence between two components given the remaining components to identify the edges of the graph. As an example we consider multivariate autoregressive processes. The method is applied to air pollution data. Received: June 1999     

Heteroskedasticity-consistent covariance matrix estimators for spatial autoregressive models

In the presence of heteroskedasticity, conventional test statistics based on the ordinary least squares (OLS) estimator lead to incorrect inference results for the linear regression model. Given that heteroskedasticity is common in cross-sectional data, the test statistics based on various forms of heteroskedasticity-consistent covariance matrices (HCCMs) have been developed in the literature. In contrast to the standard linear regression model, heteroskedasticity is a more serious problem for spatial econometric models, generally causing inconsistent extremum estimators of model coefficients. This paper investigates the finite sample properties of the heteroskedasticity-robust generalized method of moments estimator (RGMME) for a spatial econometric model with an unknown form of heteroskedasticity. In particular, it develops various HCCM-type corrections to improve the finite sample properties of the RGMME and the conventional Wald test. The Monte Carlo results indicate that the HCCM-type corrections can produce more accurate results for inference on model parameters and the impact effects estimates in small samples.     

Temporal Fusion Transformers for interpretable multi-horizon time series forecasting

Multi-horizon forecasting often contains a complex mix of inputs – including static (i.e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed in the past – without any prior information on how they interact with the target. Several deep learning methods have been proposed, but they are typically ‘black-box’ models that do not shed light on how they use the full range of inputs present in practical scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) – a novel attention-based architecture that combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. To learn temporal relationships at different scales, TFT uses recurrent layers for local processing and interpretable self-attention layers for long-term dependencies. TFT utilizes specialized components to select relevant features and a series of gating layers to suppress unnecessary components, enabling high performance in a wide range of scenarios. On a variety of real-world datasets, we demonstrate significant performance improvements over existing benchmarks, and highlight three practical interpretability use cases of TFT.     

LoMEF: A framework to produce local explanations for global model time series forecasts

Global forecasting models (GFMs) that are trained across a set of multiple time series have shown superior results in many forecasting competitions and real-world applications compared with univariate forecasting approaches. One aspect of the popularity of statistical forecasting models such as ETS and ARIMA is their relative simplicity and interpretability (in terms of relevant lags, trend, seasonality, and other attributes), while GFMs typically lack interpretability, especially relating to particular time series. This reduces the trust and confidence of stakeholders when making decisions based on the forecasts without being able to understand the predictions. To mitigate this problem, we propose a novel local model-agnostic interpretability approach to explain the forecasts from GFMs. We train simpler univariate surrogate models that are considered interpretable (e.g., ETS) on the predictions of the GFM on samples within a neighbourhood that we obtain through bootstrapping, or straightforwardly as the one-step-ahead global black-box model forecasts of the time series which needs to be explained. After, we evaluate the explanations for the forecasts of the global models in both qualitative and quantitative aspects such as accuracy, fidelity, stability, and comprehensibility, and are able to show the benefits of our approach.     

Principles and algorithms for forecasting groups of time series: Locality and globality

Global methods that fit a single forecasting method to all time series in a set have recently shown surprising accuracy, even when forecasting large groups of heterogeneous time series. We provide the following contributions that help understand the potential and applicability of global methods and how they relate to traditional local methods that fit a separate forecasting method to each series:

• •Global and local methods can produce the same forecasts without any assumptions about similarity of the series in the set.
• •The complexity of local methods grows with the size of the set while it remains constant for global methods. This result supports the recent evidence and provides principles for the design of new algorithms.
• •In an extensive empirical study, we show that purposely naïve algorithms derived from these principles show outstanding accuracy. In particular, global linear models provide competitive accuracy with far fewer parameters than the simplest of local methods.

     

Estimation and model adequacy checking for multivariate seasonal autoregressive time series models with periodically varying parameters

We introduce a class of multivariate seasonal time series models with periodically varying parameters, abbreviated by the acronym SPVAR. The model is suitable for multivariate data, and combines a periodic autoregressive structure and a multiplicative seasonal time series model. The stationarity conditions (in the periodic sense) and the theoretical autocovariance functions of SPVAR stochastic processes are derived. Estimation and checking stages are considered. The asymptotic normal distribution of the least squares estimators of the model parameters is established, and the asymptotic distributions of the residual autocovariance and autocorrelation matrices in the class of SPVAR time series models are obtained. In order to check model adequacy, portmanteau test statistics are considered and their asymptotic distributions are studied. A simulation study is briefly discussed to investigate the finite-sample properties of the proposed test statistics. The methodology is illustrated with a bivariate quarterly data set on travelers entering in to Canada.     

A copula-based time series model for global horizontal irradiation

The increasing importance of solar power for electricity generation leads to increasing demand for probabilistic forecasting of local and aggregated photovoltaic (PV) yields. Based on publicly available irradiation data, this paper uses an indirect modeling approach for hourly medium to long-term local PV yields. We suggest a time series model for global horizontal irradiation that allows for multivariate probabilistic forecasts for arbitrary time horizons. It features several important stylized facts. Sharp time-dependent lower and upper bounds of global horizontal irradiations are estimated. The parameters of the beta distributed marginals of the transformed data are allowed to be time-dependent. A copula-based time series model is introduced for the hourly and daily dependence structure based on simple vine copulas with so-called tail dependence. Evaluation methods based on scoring rules are used to compare the model’s power for multivariate probabilistic forecasting with other models used in the literature showing that our model outperforms other models in many respects.     

A heuristic method for parameter selection in LS-SVM: Application to time series prediction

Least Squares Support Vector Machines (LS-SVM) are the state of the art in kernel methods for regression. These models have been successfully applied for time series modelling and prediction. A critical issue for the performance of these models is the choice of the kernel parameters and the hyperparameters which define the function to be minimized. In this paper a heuristic method for setting both the σ parameter of the Gaussian kernel and the regularization hyperparameter based on information extracted from the time series to be modelled is presented and evaluated.     

Estimation of variances in orthogonal finite discrete spectrum linear regression models

The Invariant Quadratic Estimators, the Maximum Likelihood Estimator (MLE) and Restricted Maximum Likelihood Estimator (REML) of variances in an orthogonal Finite Discrete Spectrum Linear Regression Model (FDSLRM) are derived and the problems of unbiasedness and consistency of these estimators are investigated.Acknowledgement. The research was supported by the grants 1/0272/03, 1/0264/03 and 2/4026/04 of the Slovak Scientific Grant Agency VEGA.     

Mixture of MLP-experts for trend forecasting of time series: A case study of the Tehran stock exchange

A new method for forecasting the trend of time series, based on mixture of MLP experts, is presented. In this paper, three neural network combining methods and an Adaptive Network-Based Fuzzy Inference System (ANFIS) are applied to trend forecasting in the Tehran stock exchange. There are two experiments in this study. In experiment I, the time series data are the Kharg petrochemical company’s daily closing prices on the Tehran stock exchange. In this case study, which considers different schemes for forecasting the trend of the time series, the recognition rates are 75.97%, 77.13% and 81.64% for stacked generalization, modified stacked generalization and ANFIS, respectively. Using the mixture of MLP experts (ME) scheme, the recognition rate is strongly increased to 86.35%. A gain and loss analysis is also used, showing the relative forecasting success of the ME method with and without rejection criteria, compared to a simple buy and hold approach. In experiment II, the time series data are the daily closing prices of 37 companies on the Tehran stock exchange. This experiment is conducted to verify the results of experiment I and to show the efficiency of the ME method compared to stacked generalization, modified stacked generalization and ANFIS.     

The M4 Competition: 100,000 time series and 61 forecasting methods

The M4 Competition follows on from the three previous M competitions, the purpose of which was to learn from empirical evidence both how to improve the forecasting accuracy and how such learning could be used to advance the theory and practice of forecasting. The aim of M4 was to replicate and extend the three previous competitions by: (a) significantly increasing the number of series, (b) expanding the number of forecasting methods, and (c) including prediction intervals in the evaluation process as well as point forecasts. This paper covers all aspects of M4 in detail, including its organization and running, the presentation of its results, the top-performing methods overall and by categories, its major findings and their implications, and the computational requirements of the various methods. Finally, it summarizes its main conclusions and states the expectation that its series will become a testing ground for the evaluation of new methods and the improvement of the practice of forecasting, while also suggesting some ways forward for the field.     

Endemic-epidemic models with discrete-time serial interval distributions for infectious disease prediction

Multivariate count time series models are an important tool for analyzing and predicting the spread of infectious disease. We consider the endemic-epidemic framework, a class of autoregressive models for infectious disease surveillance counts, and replace the default autoregression on counts from the previous time period with more flexible weighting schemes inspired by discrete-time serial interval distributions. We employ three different parametric formulations, each with an additional unknown weighting parameter estimated via a profile likelihood approach, and compare them to an unrestricted nonparametric approach. The new methods are illustrated in a univariate analysis of dengue fever incidence in San Juan, Puerto Rico, and a spatiotemporal study of viral gastroenteritis in the 12 districts of Berlin. We assess the predictive performance of the suggested models and several reference models at various forecast horizons. In both applications, the performance of the endemic-epidemic models is considerably improved by the proposed weighting schemes.