An ensemble approach to GEFCom2017 probabilistic load forecasting

We present an ensembling approach to medium-term probabilistic load forecasting which ranked second out of 73 competitors in the defined data track of the GEFCom2017 qualifying match. In addition to being accurate, the ensemble method is highly scalable, due to the fact that it had to be applied to nine quantiles in ten zones and for six rounds. Candidate forecasts were generated using random settings for input data, covariates, and learning algorithms. The best candidate forecasts were averaged to create the final forecast, with the number of candidate forecasts being chosen based on their accuracy in similar validation periods.     

Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting

The Global Energy Forecasting Competition 2017 (GEFCom2017) attracted more than 300 students and professionals from over 30 countries for solving hierarchical probabilistic load forecasting problems. Of the series of global energy forecasting competitions that have been held, GEFCom2017 is the most challenging one to date: the first one to have a qualifying match, the first one to use hierarchical data with more than two levels, the first one to allow the usage of external data sources, the first one to ask for real-time ex-ante forecasts, and the longest one. This paper introduces the qualifying and final matches of GEFCom2017, summarizes the top-ranked methods, publishes the data used in the competition, and presents several reflections on the competition series and a vision for future energy forecasting competitions.     

A calibration method with dynamic updates for within-match forecasting of wins in tennis

In-match predictions of player win probabilities for professional tennis matches have a wide range of potential applications, including betting, fan engagement, and performance evaluation. The ideal properties of an in-play prediction method include the ability to incorporate both useful pre-match information and relevant in-match information as the match progresses, in order to update the pre-match expectations. This paper presents an in-play forecasting method that achieves both of these goals by combining a pre-match calibration method with a dynamic empirical Bayes updating rule. We present an optimisation rule for guiding the specifications of the dynamic updates using a large sample of professional tennis matches. We apply the results to data from the 2017 season and show that the dynamic model provides a 28% reduction in the error of in-match serve predictions and improves the win prediction accuracy by four percentage points relative to a constant ability model. The method is applied to two Australian Open men’s matches, and we derive several corollary statistics to highlight key dynamics in the win probabilities during a match.     

Predicting monthly biofuel production using a hybrid ensemble forecasting methodology

This paper proposes a hybrid ensemble forecasting methodology that integrating empirical mode decomposition (EMD), long short-term memory (LSTM) and extreme learning machine (ELM) for the monthly biofuel (a typical agriculture-related energy) production based on the principle of decomposition—reconstruction—ensemble. The proposed methodology involves four main steps: data decomposition via EMD, component reconstruction via a fine-to-coarse (FTC) method, individual prediction via LSTM and ELM algorithms, and ensemble prediction via a simple addition (ADD) method. For illustration and verification, the biofuel monthly production data of the USA is used as the our sample data, and the empirical results indicate that the proposed hybrid ensemble forecasting model statistically outperforms all considered benchmark models considered in terms of the forecasting accuracy. This indicates that the proposed hybrid ensemble forecasting methodology integrating the EMD-LSTM-ELM models based on the decomposition—reconstruction—ensemble principle has been proved to be a competitive model for the prediction of biofuel production.     

Empirical prediction intervals revisited

Empirical prediction intervals are constructed based on the distribution of previous out-of-sample forecast errors. Given historical data, a sample of such forecast errors is generated by successively applying a chosen point forecasting model to a sequence of fixed windows of past observations and recording the associated deviations of the model predictions from the actual observations out-of-sample. The suitable quantiles of the distribution of these forecast errors are then used along with the point forecast made by the selected model to construct an empirical prediction interval. This paper re-examines the properties of the empirical prediction interval. Specifically, we provide conditions for its asymptotic validity, evaluate its small sample performance and discuss its limitations.     

Neural networks for GEFCom2017 probabilistic load forecasting

This report describes the forecasting model which was developed by team “4C” for the global energy forecasting competition 2017 (GEFCom2017), with some modifications added afterwards to improve its accuracy. The model is based on neural networks. Temperature scenarios obtained from historical data are used as inputs to the neural networks in order to create load scenarios, and these load scenarios are then transformed into quantiles. By using a feature selection approach that is based on a stepwise regression technique, a neural network based model is developed for each zone. Furthermore, a dynamic choice of the temperature scenarios is suggested. The feature selection and dynamic choice of the temperature scenarios can improve the quantile scores considerably, resulting in very accurate forecasts among the top teams.     

Comparing probabilistic forecasts of the daily minimum and maximum temperature

Understanding changes in the frequency, severity, and seasonality of daily temperature extremes is important for public policy decisions regarding heat waves and cold snaps. A heat wave is sometimes defined in terms of both the daily minimum and maximum temperature, which necessitates the generation of forecasts of their joint distribution. In this paper, we develop time series models with the aim of providing insight and producing forecasts of the joint distribution that can challenge the accuracy of forecasts based on ensemble predictions from a numerical weather prediction model. We use ensemble model output statistics to recalibrate the raw ensemble predictions for the marginal distributions, with ensemble copula coupling used to capture the dependency between the marginal distributions. In terms of time series modelling, we consider a bivariate VARMA-MGARCH model. We use daily Spanish data recorded over a 65-year period, and find that, for the 5-year out-of-sample period, the recalibrated ensemble predictions outperform the time series models in terms of forecast accuracy.     

Combining predictive distributions for the statistical post-processing of ensemble forecasts

Statistical post-processing techniques are now used widely for correcting systematic biases and errors in the calibration of ensemble forecasts obtained from multiple runs of numerical weather prediction models. A standard approach is the ensemble model output statistics (EMOS) method, which results in a predictive distribution that is given by a single parametric law, with parameters that depend on the ensemble members. This article assesses the merits of combining multiple EMOS models based on different parametric families. In four case studies with wind speed and precipitation forecasts from two ensemble prediction systems, we investigate the performances of state of the art forecast combination methods and propose a computationally efficient approach for determining linear pool combination weights. We study the performance of forecast combination compared to that of the theoretically superior but cumbersome estimation of a full mixture model, and assess which degree of flexibility of the forecast combination approach yields the best practical results for post-processing applications.     

Accuracy of German federal election forecasts, 2013 & 2017

The present study reviews the accuracy of four methods (polls, prediction markets, expert judgment, and quantitative models) for forecasting the two German federal elections in 2013 and 2017. On average across both elections, polls and prediction markets were most accurate, while experts and quantitative models were least accurate. However, the accuracy of individual forecasts did not correlate across elections. That is, the methods that were most accurate in 2013 did not perform particularly well in 2017. A combined forecast, calculated by averaging forecasts within and across methods, was more accurate than three of the four component forecasts. The results conform to prior research on US presidential elections in showing that combining is effective in generating accurate forecasts and avoiding large errors.     

Probabilistic wind forecasting up to three months ahead using ensemble predictions for geopotential height

We develop a method for forecasting the distribution of the daily surface wind speed at timescales from 15-days to 3-months in France. On such long-term timescales, ensemble predictions of the surface wind speed have poor performance, however, the wind speed distribution may be related to the large-scale circulation of the atmosphere, for which the ensemble forecasts have better skill. The information from the large-scale circulation, represented by the 500 hPa geopotential height, is summarized into a single index by first running a PCA and then a polynomial regression. We estimate, over 20 years of daily data, the conditional probability density of the wind speed at a specific location given the index. We then use the ECMWF seasonal forecast ensemble to predict the index for horizons from 15-days to 3-months. These predictions are plugged into the conditional density to obtain a distributional forecast of surface wind. These probabilistic forecasts remain sharper than the climatology up to 1-month forecast horizon. Using a statistical postprocessing method to recalibrate the ensemble leads to further improvement of our probabilistic forecast, which then remains calibrated and sharper than the climatology up to 3-months horizon, particularly in the north of France in winter and fall.     

Exploiting sports-betting market using machine learning

We introduce a forecasting system designed to profit from sports-betting market using machine learning. We contribute three main novel ingredients. First, previous attempts to learn models for match-outcome prediction maximized the model’s predictive accuracy as the single criterion. Unlike these approaches, we also reduce the model’s correlation with the bookmaker’s predictions available through the published odds. We show that such an optimized model allows for better profit generation, and the approach is thus a way to ‘exploit’ the bookmaker. The second novelty is in the application of convolutional neural networks for match outcome prediction. The convolution layer enables to leverage a vast number of player-related statistics on its input. Thirdly, we adopt elements of the modern portfolio theory to design a strategy for bet distribution according to the odds and model predictions, trading off profit expectation and variance optimally. These three ingredients combine towards a betting method yielding positive cumulative profits in experiments with NBA data from seasons 2007–2014 systematically, as opposed to alternative methods tested.     

Robust recurrent network model for intermittent time-series forecasting

This paper describes a deep-learning-based time-series forecasting method that was ranked third in the accuracy challenge of the M5 competition. We solved the problem using a deep-learning approach based on DeepAR, which is an auto-regressive recurrent network model conditioned on historical inputs. To address the intermittent and irregular characteristics of sales demand, we modified the training procedure of DeepAR; instead of using actual values for the historical inputs, our model uses values sampled from a trained distribution and feeds them to the network as past values. We obtained the final result using an ensemble of multiple models to make a robust and stable prediction. To appropriately select a model for the ensemble, each model was evaluated using the average weighted root mean squared scaled error, calculated for all levels of a wide range of past periods.     

The M5 uncertainty competition: Results,findings and conclusions

This paper describes the M5 “Uncertainty” competition, the second of two parallel challenges of the latest M competition, aiming to advance the theory and practice of forecasting. The particular objective of the M5 “Uncertainty” competition was to accurately forecast the uncertainty distributions of the realized values of 42,840 time series that represent the hierarchical unit sales of the largest retail company in the world by revenue, Walmart. To do so, the competition required the prediction of nine different quantiles (0.005, 0.025, 0.165, 0.250, 0.500, 0.750, 0.835, 0.975, and 0.995), that can sufficiently describe the complete distributions of future sales. The paper provides details on the implementation and execution of the M5 “Uncertainty” competition, presents its results and the top-performing methods, and summarizes its major findings and conclusions. Finally, it discusses the implications of its findings and suggests directions for future research.     

A profitable model for predicting the over/under market in football

The over/under 2.5 goals betting market allows gamblers to bet on whether the total number of goals in a football match will exceed 2.5. In this paper, a set of ratings, named ‘Generalised Attacking Performance’ (GAP) ratings, are defined which measure the attacking and defensive performance of each team in a league. GAP ratings are used to forecast matches in ten European football leagues and their profitability is tested in the over/under market using two value betting strategies. GAP ratings with match statistics such as shots and shots on target as inputs are shown to yield better predictive value than the number of goals. An average profit of around 0.8 percent per bet taken is demonstrated over twelve years when using only shots and corners (and not goals) as inputs. The betting strategy is shown to be robust by comparing it to a random betting strategy.     

Endogenous qualifications and firms'' agglomeration

This paper analyzes firms' location when workers endogenously choose to qualify for professional skills but when they remain uncertain about the potential match between their personal abilities and/or affinities and the firms' specific production tasks. By qualifying in a region where firms agglomerate, workers benefit from higher prospects of good match. At the equilibrium, we show that firms may locate in a single cluster, symmetric clusters or even asymmetric clusters. Comparative statics with respect to product market demand and labor supply parameters are provided.     

Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States

The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden of COVID-19 in the United States from many contributing teams. We study methods for building an ensemble that combines forecasts from these teams. These experiments have informed the ensemble methods used by the Hub. To be most useful to policymakers, ensemble forecasts must have stable performance in the presence of two key characteristics of the component forecasts: (1) occasional misalignment with the reported data, and (2) instability in the relative performance of component forecasters over time. Our results indicate that in the presence of these challenges, an untrained and robust approach to ensembling using an equally weighted median of all component forecasts is a good choice to support public health decision-makers. In settings where some contributing forecasters have a stable record of good performance, trained ensembles that give those forecasters higher weight can also be helpful.     

A solution for M5 Forecasting - Uncertainty: Hybrid gradient boosting and autoregressive recurrent neural network for quantile estimation

This paper presents our 13th place solution to the M5 Forecasting - Uncertainty challenge and compares it against GoodsForecast’s second-place solution. This challenge aims to estimate the median and eight other quantiles of various product sales in Walmart. Both solutions handle the predictions of median and other quantiles separately. Our solution hybridizes LightGBM and DeepAR in various ways for median and quantile estimation, based on the aggregation levels of the sales. Similarly, GoodsForecast’s solution also utilized a hybrid approach, i.e., LightGBM for point estimation and a Histogram algorithm for quantile estimation. In this paper, the differences between the two solutions and their results are highlighted. Despite our solution only taking 13th place in the challenge with the competition metric, it achieves the lowest average rank based on the multiple comparisons with the best (MCB) test which implies the most accurate forecasts in the majority of the series. It also indicates better performance at the product-store aggregation level which comprises 30,490 (71.2% of all) series compared to most teams.     

A decomposition clustering ensemble learning approach for forecasting foreign exchange rates

A decomposition clustering ensemble (DCE) learning approach is proposed for forecasting foreign exchange rates by integrating the variational mode decomposition (VMD), the self-organizing map (SOM) network, and the kernel extreme learning machine (KELM). First, the exchange rate time series is decomposed into N subcomponents by the VMD method. Second, each subcomponent series is modeled by the KELM. Third, the SOM neural network is introduced to cluster the subcomponent forecasting results of the in-sample dataset to obtain cluster centers. Finally, each cluster's ensemble weight is estimated by another KELM, and the final forecasting results are obtained by the corresponding clusters' ensemble weights. The empirical results illustrate that our proposed DCE learning approach can significantly improve forecasting performance, and statistically outperform some other benchmark models in directional and level forecasting accuracy.     

基于集成学习的多式联运货运量预测模型构建

货运量精准预测是多式联运网络高效协同发展的重要基础,货运量时变性强、数据多样性缺失是实现精准货运量预测的问题所在。基于此,通过挖掘货物运输量(集装箱)的时间变化特征,构建初始相关时间特征输入集,结合斯皮尔曼相关性系数分布,采用Bagging+BP集成学习方法训练多个弱分类器,最终组合获取高精度的强学习模型。以南京龙潭港为例,对自回归移动平均模型(ARIMA)、Bagging+BP集成学习网络以及长短时记忆神经网络(LSTM)三种模型进行评价,实验结果表明,相比于其他模型,提出的Bagging+BP集成学习网络预测性能良好,有一定的实用价值。     

Data preprocessing and quantile regression for probabilistic load forecasting in the GEFCom2017 final match

Team QUINKAN competed in the GEFCom2017 final match of hierarchical probabilistic load forecasting by adopting the quantile regression method using the R package quantreg. The weather stations were clustered into 11 groups, from which an optimal one was chosen for each load meter using the boosting method. The load meter records were cleaned and/or supplemented by various methods in order to secure robust quantile predictions. The variation in the regression formulas was kept as small as possible by introducing measures for suppressing prediction instability, although special formulas were employed for loading meters that were of an industrial nature. Several procedures were applied to help improve the accuracy, such as the smoothing of season transitions, coarse graining of the relative humidity, the use of load-oriented day-type definition, the averaging of weather data, and outlier removal.