DeepAR: Probabilistic forecasting with autoregressive recurrent networks

Probabilistic forecasting, i.e., estimating a time series’ future probability distribution given its past, is a key enabler for optimizing business processes. In retail businesses, for example, probabilistic demand forecasts are crucial for having the right inventory available at the right time and in the right place. This paper proposes DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an autoregressive recurrent neural network model on a large number of related time series. We demonstrate how the application of deep learning techniques to forecasting can overcome many of the challenges that are faced by widely-used classical approaches to the problem. By means of extensive empirical evaluations on several real-world forecasting datasets, we show that our methodology produces more accurate forecasts than other state-of-the-art methods, while requiring minimal manual work.     

Probabilistic wind forecasting up to three months ahead using ensemble predictions for geopotential height

We develop a method for forecasting the distribution of the daily surface wind speed at timescales from 15-days to 3-months in France. On such long-term timescales, ensemble predictions of the surface wind speed have poor performance, however, the wind speed distribution may be related to the large-scale circulation of the atmosphere, for which the ensemble forecasts have better skill. The information from the large-scale circulation, represented by the 500 hPa geopotential height, is summarized into a single index by first running a PCA and then a polynomial regression. We estimate, over 20 years of daily data, the conditional probability density of the wind speed at a specific location given the index. We then use the ECMWF seasonal forecast ensemble to predict the index for horizons from 15-days to 3-months. These predictions are plugged into the conditional density to obtain a distributional forecast of surface wind. These probabilistic forecasts remain sharper than the climatology up to 1-month forecast horizon. Using a statistical postprocessing method to recalibrate the ensemble leads to further improvement of our probabilistic forecast, which then remains calibrated and sharper than the climatology up to 3-months horizon, particularly in the north of France in winter and fall.     

Interpreting the skill score form of forecast performance metrics

Performance measures of point forecasts are expressed commonly as skill scores, in which the performance gain from using one forecasting system over another is expressed as a proportion of the gain achieved by forecasting that outcome perfectly. Increasingly, it is common to express scores of probabilistic forecasts in this form; however, this paper presents three criticisms of this approach. Firstly, initial condition uncertainty (which is outside the forecaster’s control) limits the capacity to improve a probabilistic forecast, and thus a ‘perfect’ score is often unattainable. Secondly, the skill score forms of the ignorance and Brier scores are biased. Finally, it is argued that the skill score form of scoring rules destroys the useful interpretation in terms of the relative skill levels of two forecasting systems. Indeed, it is often misleading, and useful information is lost when the skill score form is used in place of the original score.     

Probabilistic forecasting of industrial electricity load with regime switching behavior

This paper suggests a novel inhomogeneous Markov switching approach for the probabilistic forecasting of industrial companies’ electricity loads, for which the load switches at random times between production and standby regimes. The model that we propose describes the transitions between the regimes using a hidden Markov chain with time-varying transition probabilities that depend on calendar variables. We model the demand during the production regime using an autoregressive moving-average (ARMA) process with seasonal patterns, whereas we use a much simpler model for the standby regime in order to reduce the complexity. The maximum likelihood estimation of the parameters is implemented using a differential evolution algorithm. Using the continuous ranked probability score (CRPS) to evaluate the goodness-of-fit of our model for probabilistic forecasting, it is shown that this model often outperforms classical additive time series models, as well as homogeneous Markov switching models. We also propose a simple procedure for classifying load profiles into those with and without regime-switching behaviors.     

Evaluating quantile-bounded and expectile-bounded interval forecasts

In many different contexts, decision-making is improved by the availability of probabilistic predictions. The accuracy of probabilistic forecasting methods can be compared using scoring functions and insight provided by calibration tests. These tests evaluate the consistency of predictions with the observations. Our main agenda in this paper is interval forecasts and their evaluation. Such forecasts are usually bounded by two quantile forecasts. However, a limitation of quantiles is that they convey no information regarding the size of potential exceedances. By contrast, the location of an expectile is dictated by the whole distribution. This prompts us to propose expectile-bounded intervals. We provide interpretation, a consistent scoring function and a calibration test. Before doing this, we reflect on the evaluation of forecasts of quantile-bounded intervals and expectiles, and suggest extensions of previously proposed calibration tests in order to guard against strategic forecasting. We illustrate ideas using day-ahead electricity price forecasting.     

Price forecasting and trigger price probability estimation

This paper shows how probability estimation, incorporated within price prediction from a single equation, will enhance the usefulness of forecast information and will have greater intuitive appeal. The estimated probability is the probability that the price will cross some predetermined threshold of importance, a trigger price. This procedure is likely to be more useful in economic decision making than the application of prediction interval estimates since the latter approach does not convey threshold probability information. Empirical application encompasses forecasting in the watermelon production industry to demonstrate the power and appeal of the approach. All probability estimates were more precise than a 0.5 probability of occurence.     

Enhancing capacity planning through forecasting: An integrated tool for maintenance of complex product systems

Despite the extensive amount of data generated and stored during the maintenance capacity planning process, Maintenance, Repair, and Overhaul (MRO) organizations have yet to explore their full potential in forecasting the required capacity to face future and unprecedented maintenance interventions. This paper explores the integration of time series forecasting capabilities in a tool for maintenance capacity planning of complex product systems (CoPS), intended to value data that is routinely generated and stored, but often disregarded by MROs. State space formulations with multiplicative errors for the simple exponential smoothing (SES), Holt’s linear method (HLM), additive Holt-Winters (AHW), and multiplicative Holt-Winters (MHW) are assessed using real data, comprised of 171 maintenance projects collected from a major Portuguese aircraft MRO. A state space formulation of the MHW, selected using the bias-corrected Akaike information criterion (AICc), is integrated in a Decision Support System (DSS) for capacity planning with probabilistic inference capabilities and used to forecast the workload probability distribution of a future and unprecedent maintenance intervention. The developed tool is validated by comparing forecasted values with workloads of a particular maintenance intervention and with a model simulating current forecasting practices employed by MROs.     

Probabilistic electricity price forecasting with NARX networks: Combine point or probabilistic forecasts?

Recent electricity price forecasting studies have shown that decomposing a series of spot prices into a long-term trend-seasonal and a stochastic component, modeling them independently and then combining their forecasts, can yield more accurate point predictions than an approach in which the same regression or neural network model is calibrated to the prices themselves. Here, considering two novel extensions of this concept to probabilistic forecasting, we find that (i) efficiently calibrated non-linear autoregressive with exogenous variables (NARX) networks can outperform their autoregressive counterparts, even without combining forecasts from many runs, and that (ii) in terms of accuracy it is better to construct probabilistic forecasts directly from point predictions. However, if speed is a critical issue, running quantile regression on combined point forecasts (i.e., committee machines) may be an option worth considering. Finally, we confirm an earlier observation that averaging probabilities outperforms averaging quantiles when combining predictive distributions in electricity price forecasting.     

A comparison of the accuracy of the Box-Jenkins method with that of automated forecasting methods

The focus of the research described in this paper is on presenting an automated forecasting system that encompasses an objective ARIMA method with the Holt-Winters procedure in a weighted averaging scheme. The system is applied to M-Competition data and the results are compared to the subjective Box-Jenkins forecasts as well as to results from two other automated methods, CAPRI and SIFT. The comparison reveals that especially for one-step ahead forecasting, the automated system competes favorably with both automated methods and the individualized Box-Jenkins analysis.     

Forecasting risk measures using intraday data in a generalized autoregressive score framework

A new framework for the joint estimation and forecasting of dynamic value at risk (VaR) and expected shortfall (ES) is proposed by our incorporating intraday information into a generalized autoregressive score (GAS) model introduced by Patton et al., 2019 to estimate risk measures in a quantile regression set-up. We consider four intraday measures: the realized volatility at 5-min and 10-min sampling frequencies, and the overnight return incorporated into these two realized volatilities. In a forecasting study, the set of newly proposed semiparametric models are applied to four international stock market indices (S&P 500, Dow Jones Industrial Average, Nikkei 225 and FTSE 100) and are compared with a range of parametric, nonparametric and semiparametric models, including historical simulations, generalized autoregressive conditional heteroscedasticity (GARCH) models and the original GAS models. VaR and ES forecasts are backtested individually, and the joint loss function is used for comparisons. Our results show that GAS models, enhanced with the realized volatility measures, outperform the benchmark models consistently across all indices and various probability levels.     

Pitfalls in simulation-based evaluation of forecast monitoring schemes

To evaluate the performance of a forecast monitoring scheme, forecasters have traditionally used a simulation-based estimator of some characteristic of the associated run length distribution. The most frequently cited performance measures are the average run length and the probability that the run length does not exceed a user-specified cutoff point. However, there is disagreement about the definition of run length that is appropriate in the context of forecasting. In this note we present the precise relationships between conflicting formulations both of the average run length and of the probability distribution function for the run length. The practical significance of each of these relationships is discussed. These results bear directly on the way in which simulation experiments should be designed and executed to compare alternative monitoring schemes.     

Forecasting U.S. money growth using economic uncertainty measures and regularisation techniques

This paper examines the out-of-sample forecasting properties of six different economic uncertainty variables for the growth of the real M2 and real M4 Divisia money series for the U.S. using monthly data. The core contention is that information on economic uncertainty improves the forecasting accuracy. We estimate vector autoregressive models using the iterated rolling-window forecasting scheme, in combination with modern regularisation techniques from the field of machine learning. Applying the Hansen-Lunde-Nason model confidence set approach under two different loss functions reveals strong evidence that uncertainty variables that are related to financial markets, the state of the macroeconomy or economic policy provide additional informational content when forecasting monetary dynamics. The use of regularisation techniques improves the forecast accuracy substantially.     

M5 competition uncertainty: Overdispersion,distributional forecasting,GAMLSS, and beyond

The M5 competition uncertainty track aims for probabilistic forecasting of sales of thousands of Walmart retail goods. We show that the M5 competition data face strong overdispersion and sporadic demand, especially zero demand. We discuss modeling issues concerning adequate probabilistic forecasting of such count data processes. Unfortunately, the majority of popular prediction methods used in the M5 competition (e.g. lightgbm and xgboost GBMs) fail to address the data characteristics, due to the considered objective functions. Distributional forecasting provides a suitable modeling approach to overcome those problems. The GAMLSS framework allows for flexible probabilistic forecasting using low-dimensional distributions. We illustrate how the GAMLSS approach can be applied to M5 competition data by modeling the location and scale parameters of various distributions, e.g. the negative binomial distribution. Finally, we discuss software packages for distributional modeling and their drawbacks, like the R package gamlss with its package extensions, and (deep) distributional forecasting libraries such as TensorFlow Probability.     

Forecasting correlated time series with exponential smoothing models

This paper presents the Bayesian analysis of a general multivariate exponential smoothing model that allows us to forecast time series jointly, subject to correlated random disturbances. The general multivariate model, which can be formulated as a seemingly unrelated regression model, includes the previously studied homogeneous multivariate Holt-Winters’ model as a special case when all of the univariate series share a common structure. MCMC simulation techniques are required in order to approach the non-analytically tractable posterior distribution of the model parameters. The predictive distribution is then estimated using Monte Carlo integration. A Bayesian model selection criterion is introduced into the forecasting scheme for selecting the most adequate multivariate model for describing the behaviour of the time series under study. The forecasting performance of this procedure is tested using some real examples.     

Nonparametric search

We analyze the search problem of a consumer who derives information only from the sequential search process. This paper considers the case of a consumer who uses a nonparametric procedure to estimate the probability distribution. It is shown that a solution to the consumer's problem is a very simple strategy which depends only on the order statistics, on the discounting factor, and on the duration of the search. It leads to a finite search almost surely. This optimal strategy is a myopic rule which is computable and which is characterized by a sequence of strictly increasing reservation prices.     

An economic–probabilistic model for risk analysis in technological innovation projects

This paper presents the application of an economic–probabilistic model to conduct risk analysis in technological innovation (TI) projects. The model integrates risk and economic analysis by quantifying both value and probability of occurrence of cash flow deviations, thus resulting in an economic–probabilistic analysis of the expected returns. The main risk categories and factors in TI projects are identified and associated to cash flow groups. The model allows to calculate risk-adjusted values for cash flow groups and project net present value through stochastic simulation. As a result, the model provides both the risk-adjusted project economic return with the associated probability distribution to its NPV and the variability that each risk factor generates in the project return. The model offers important benefits from the point of view of practitioners, including a condensed list of independent risk factors and the use of a monetary scale to assess risk impact which is familiar to most decision makers.     

A Bayesian learning procedure for the.(s, Q) inventory policy

We present an asymptotically optimal Bayesian learning procedure for the ( s, Q ) inventory policy, for the case when the probability distribution of lead time demand is unknown. This distribution is not required to be a member of a certain family, and the maximal lead time demand is also allowed to be unknown. The algorithm developed for this purpose Is an extension of a standard iterative procedure, which in its original form -in spite of claims to the contrary-might produce solution values that are arbitrarily far away from the optimal one.     

FIML estimation of the dynamic simultaneous equations model with ARMA disturbances

A method is presented for the estimation of the parameters in the dynamic simultaneous equations model with vector autoregressive moving average disturbances. The estimation procedure is derived from the full information maximum likelihood approach and is based on Newton-Raphson techniques applied to the likelihood equations. The resulting two-step Newton-Raphson procedure involves only generalized instrumental variables estimation in the second step. This procedure also serves as the basis for an iterative scheme to solve the normal equations and obtain the maximum likelihood estimates of the conditional likelihood function. A nine-equation variant of the quarterly forecasting model of the US economy developed by Fair is then used as a realistic example to illustrate the estimation procedure described in the paper.     

Approximate Bayesian forecasting

Approximate Bayesian Computation (ABC) has become increasingly prominent as a method for conducting parameter inference in a range of challenging statistical problems, most notably those characterized by an intractable likelihood function. In this paper, we focus on the use of ABC not as a tool for parametric inference, but as a means of generating probabilistic forecasts; or for conducting what we refer to as ‘approximate Bayesian forecasting’. The four key issues explored are: (i) the link between the theoretical behavior of the ABC posterior and that of the ABC-based predictive; (ii) the use of proper scoring rules to measure the (potential) loss of forecast accuracy when using an approximate rather than an exact predictive; (iii) the performance of approximate Bayesian forecasting in state space models; and (iv) the use of forecasting criteria to inform the selection of ABC summaries in empirical settings. The primary finding of the paper is that ABC can provide a computationally efficient means of generating probabilistic forecasts that are nearly identical to those produced by the exact predictive, and in a fraction of the time required to produce predictions via an exact method.     

Forecast combinations for value at risk and expected shortfall

Combining provides a pragmatic way of synthesising the information provided by individual forecasting methods. In the context of forecasting the mean, numerous studies have shown that combining often leads to improvements in accuracy. Despite the importance of the value at risk (VaR), though, few papers have considered quantile forecast combinations. One risk measure that is receiving an increasing amount of attention is the expected shortfall (ES), which is the expectation of the exceedances beyond the VaR. There have been no previous studies on combining ES predictions, presumably due to there being no suitable loss function for ES. However, it has been shown recently that a set of scoring functions exist for the joint estimation or backtesting of VaR and ES forecasts. We use such scoring functions to estimate combining weights for VaR and ES prediction. The results from five stock indices show that combining outperforms the individual methods for the 1% and 5% probability levels.