Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting

The Global Energy Forecasting Competition 2017 (GEFCom2017) attracted more than 300 students and professionals from over 30 countries for solving hierarchical probabilistic load forecasting problems. Of the series of global energy forecasting competitions that have been held, GEFCom2017 is the most challenging one to date: the first one to have a qualifying match, the first one to use hierarchical data with more than two levels, the first one to allow the usage of external data sources, the first one to ask for real-time ex-ante forecasts, and the longest one. This paper introduces the qualifying and final matches of GEFCom2017, summarizes the top-ranked methods, publishes the data used in the competition, and presents several reflections on the competition series and a vision for future energy forecasting competitions.     

Neural networks for GEFCom2017 probabilistic load forecasting

This report describes the forecasting model which was developed by team “4C” for the global energy forecasting competition 2017 (GEFCom2017), with some modifications added afterwards to improve its accuracy. The model is based on neural networks. Temperature scenarios obtained from historical data are used as inputs to the neural networks in order to create load scenarios, and these load scenarios are then transformed into quantiles. By using a feature selection approach that is based on a stepwise regression technique, a neural network based model is developed for each zone. Furthermore, a dynamic choice of the temperature scenarios is suggested. The feature selection and dynamic choice of the temperature scenarios can improve the quantile scores considerably, resulting in very accurate forecasts among the top teams.     

Machine learning methods for GEFCom2017 probabilistic load forecasting

This paper describes the preprocessing and forecasting methods used by team Orbuculum during the qualifying match of the Global Energy Forecasting Competition 2017 (GEFCom2017). Tree-based algorithms (gradient boosting and quantile random forest) and neural networks made up an ensemble. The ensemble prediction quantiles were obtained by a simple averaging of the ensemble members’ prediction quantiles. The result shows a robust performance according to the pinball loss metric, with the ensemble model achieving third place in the qualifying match of the competition.     

An ensemble approach to GEFCom2017 probabilistic load forecasting

We present an ensembling approach to medium-term probabilistic load forecasting which ranked second out of 73 competitors in the defined data track of the GEFCom2017 qualifying match. In addition to being accurate, the ensemble method is highly scalable, due to the fact that it had to be applied to nine quantiles in ten zones and for six rounds. Candidate forecasts were generated using random settings for input data, covariates, and learning algorithms. The best candidate forecasts were averaged to create the final forecast, with the number of candidate forecasts being chosen based on their accuracy in similar validation periods.     

Data visualization and forecast combination for probabilistic load forecasting in GEFCom2017 final match

This paper describes the methods used by Team Cassandra, a joint effort between IBM Research Australia and the University of Melbourne, in the GEFCom2017 load forecasting competition. An important first phase in the forecasting effort involved a deep exploration of the underlying dataset. Several data visualisation techniques were applied to help us better understand the nature and size of gaps, outliers, the relationships between different entities in the dataset, and the relevance of custom date ranges. Improved, cleaned data were then used to train multiple probabilistic forecasting models. These included a number of standard and well-known approaches, as well as a neural-network based quantile forecast model that was developed specifically for this dataset. Finally, model selection and forecast combination were used to choose a custom forecasting model for every entity in the dataset.     

Comparison of intraday probabilistic forecasting of solar irradiance using only endogenous data

Accurate solar forecasts are necessary to improve the integration of solar renewables into the energy grid. In recent years, numerous methods have been developed for predicting the solar irradiance or the output of solar renewables. By definition, a forecast is uncertain. Thus, the models developed predict the mean and the associated uncertainty. Comparisons are therefore necessary and useful for assessing the skill and accuracy of these new methods in the field of solar energy.The aim of this paper is to present a comparison of various models that provide probabilistic forecasts of the solar irradiance within a very strict framework. Indeed, we consider focusing on intraday forecasts, with lead times ranging from 1 to 6 h. The models selected use only endogenous inputs for generating the forecasts. In other words, the only inputs of the models are the past solar irradiance data. In this context, the most common way of generating the forecasts is to combine point forecasting methods with probabilistic approaches in order to provide prediction intervals for the solar irradiance forecasts. For this task, we selected from the literature three point forecasting models (recursive autoregressive and moving average (ARMA), coupled autoregressive and dynamical system (CARDS), and neural network (NN)), and seven methods for assessing the distribution of their error (linear model in quantile regression (LMQR), weighted quantile regression (WQR), quantile regression neural network (QRNN), recursive generalized autoregressive conditional heteroskedasticity (GARCHrls), sieve bootstrap (SB), quantile regression forest (QRF), and gradient boosting decision trees (GBDT)), leading to a comparison of 20 combinations of models.None of the model combinations clearly outperform the others; nevertheless, some trends emerge from the comparison. First, the use of the clear sky index ensures the accuracy of the forecasts. This derived parameter permits time series to be deseasonalized with missing data, and is also a good explanatory variable of the distribution of the forecasting errors. Second, regardless of the point forecasting method used, linear models in quantile regression, weighted quantile regression and gradient boosting decision trees are able to forecast the prediction intervals accurately.     

The effects of age structure on economic growth: An application of probabilistic forecasting to India

During recent years there has been an increasing awareness of the explanatory power of population age structure variables in economic growth regressions. We estimate a new cross-country regression model of the effects of age structure change on economic growth. We use the new model and recent probabilistic demographic forecasts for India to derive the uncertainty of predicted economic growth rates caused by the uncertainty in demographic developments.     

Quantile regression for the qualifying match of GEFCom2017 probabilistic load forecasting

We present a simple quantile regression-based forecasting method that was applied in the probabilistic load forecasting framework of the Global Energy Forecasting Competition 2017 (GEFCom2017). The hourly load data are log transformed and split into a long-term trend component and a remainder term. The key forecasting element is the quantile regression approach for the remainder term, which takes into account both weekly and annual seasonalities, such as their interactions. Temperature information is used only for stabilizing the forecast of the long-term trend component. Information on public holidays is ignored. However, the forecasting method still placed second in the open data track and fourth in the definite data track, which is remarkable given the simplicity of the model. The method also outperforms the Vanilla benchmark consistently.     

DeepAR: Probabilistic forecasting with autoregressive recurrent networks

Probabilistic forecasting, i.e., estimating a time series’ future probability distribution given its past, is a key enabler for optimizing business processes. In retail businesses, for example, probabilistic demand forecasts are crucial for having the right inventory available at the right time and in the right place. This paper proposes DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an autoregressive recurrent neural network model on a large number of related time series. We demonstrate how the application of deep learning techniques to forecasting can overcome many of the challenges that are faced by widely-used classical approaches to the problem. By means of extensive empirical evaluations on several real-world forecasting datasets, we show that our methodology produces more accurate forecasts than other state-of-the-art methods, while requiring minimal manual work.     

Text-based crude oil price forecasting: A deep learning approach

This study proposes a new, novel crude oil price forecasting method based on online media text mining, with the aim of capturing the more immediate market antecedents of price fluctuations. Specifically, this is an early attempt to apply deep learning techniques to crude oil forecasting, and to extract hidden patterns within online news media using a convolutional neural network (CNN). While the news-text sentiment features and the features extracted by the CNN model reveal significant relationships with the price change, they need to be grouped according to their topics in the price forecasting in order to obtain a greater forecasting accuracy. This study further proposes a feature grouping method based on the Latent Dirichlet Allocation (LDA) topic model for distinguishing effects from various online news topics. Optimized input variable combination is constructed using lag order selection and feature selection methods. Our empirical results suggest that the proposed topic-sentiment synthesis forecasting models perform better than the older benchmark models. In addition, text features and financial features are shown to be complementary in producing more accurate crude oil price forecasts.     

Short term load forecasting and the effect of temperature at the low voltage level

Short term load forecasts will play a key role in the implementation of smart electricity grids. They are required for optimising a wide range of potential network solutions on the low voltage (LV) grid, including the integration of low carbon technologies (such as photovoltaics) and the utilisation of battery storage devices. Despite the need for accurate LV level load forecasts, much of the literature has focused on the individual household or building level using data from smart meters, or on aggregates of such data. This study provides a detailed analysis of several state-of-the-art methods for both point and probabilistic LV load forecasts. We evaluate the out-of-sample forecast accuracies of these methodologies on 100 real LV feeders, for horizons from one to four days ahead. In addition, we also test the effect of the temperature (both actual and forecast) on the accuracy of load forecasts. We present some important results on the drivers of forecasts accuracy as well as on the empirical comparison of point and probabilistic forecast measures.     

The M4 Competition: 100,000 time series and 61 forecasting methods

The M4 Competition follows on from the three previous M competitions, the purpose of which was to learn from empirical evidence both how to improve the forecasting accuracy and how such learning could be used to advance the theory and practice of forecasting. The aim of M4 was to replicate and extend the three previous competitions by: (a) significantly increasing the number of series, (b) expanding the number of forecasting methods, and (c) including prediction intervals in the evaluation process as well as point forecasts. This paper covers all aspects of M4 in detail, including its organization and running, the presentation of its results, the top-performing methods overall and by categories, its major findings and their implications, and the computational requirements of the various methods. Finally, it summarizes its main conclusions and states the expectation that its series will become a testing ground for the evaluation of new methods and the improvement of the practice of forecasting, while also suggesting some ways forward for the field.     

U-Convolutional model for spatio-temporal wind speed forecasting

The increasing penetration of intermittent renewable energy in power systems brings operational challenges. One way of supporting them is by enhancing the predictability of renewables through accurate forecasting. Convolutional Neural Networks (Convnets) provide a successful technique for processing space-structured multi-dimensional data. In our work, we propose the U-Convolutional model to predict hourly wind speeds for a single location using spatio-temporal data with multiple explanatory variables as an input. The U-Convolutional model is composed of a U-Net part, which synthesizes input information, and a Convnet part, which maps the synthesized data into a single-site wind prediction. We compare our approach with advanced Convnets, a fully connected neural network, and univariate models. We use time series from the Climate Forecast System Reanalysis as datasets and select temperature and u- and v-components of wind as explanatory variables. The proposed models are evaluated at multiple locations (totaling 181 target series) and multiple forecasting horizons. The results indicate that our proposal is promising for spatio-temporal wind speed prediction, with results that show competitive performance on both time horizons for all datasets.     

Daily retail demand forecasting using machine learning with emphasis on calendric special days

Demand forecasting is an important task for retailers as it is required for various operational decisions. One key challenge is to forecast demand on special days that are subject to vastly different demand patterns than on regular days. We present the case of a bakery chain with an emphasis on special calendar days, for which we address the problem of forecasting the daily demand for different product categories at the store level. Such forecasts are an input for production and ordering decisions. We treat the forecasting problem as a supervised machine learning task and provide an evaluation of different methods, including artificial neural networks and gradient-boosted decision trees. In particular, we outline and discuss the possibility of formulating a classification instead of a regression problem. An empirical comparison with established approaches reveals the superiority of machine learning methods, while classification-based approaches outperform regression-based approaches. We also found that machine learning methods not only provide more accurate forecasts but are also more suitable for applications in a large-scale demand forecasting scenario that often occurs in the retail industry.     

Kaggle forecasting competitions: An overlooked learning opportunity

We review the results of six forecasting competitions based on the online data science platform Kaggle, which have been largely overlooked by the forecasting community. In contrast to the M competitions, the competitions reviewed in this study feature daily and weekly time series with exogenous variables, business hierarchy information, or both. Furthermore, the Kaggle data sets all exhibit higher entropy than the M3 and M4 competitions, and they are intermittent.In this review, we confirm the conclusion of the M4 competition that ensemble models using cross-learning tend to outperform local time series models and that gradient boosted decision trees and neural networks are strong forecast methods. Moreover, we present insights regarding the use of external information and validation strategies, and discuss the impacts of data characteristics on the choice of statistics or machine learning methods. Based on these insights, we construct nine ex-ante hypotheses for the outcome of the M5 competition to allow empirical validation of our findings.     

Blending gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series

In this study, we addressed the problem of point and probabilistic forecasting by describing a blending methodology for machine learning models from the gradient boosted trees and neural networks families. These principles were successfully applied in the recent M5 Competition in both the Accuracy and Uncertainty tracks. The key points of our methodology are: (a) transforming the task into regression on sales for a single day; (b) information-rich feature engineering; (c) creating a diverse set of state-of-the-art machine learning models; and (d) carefully constructing validation sets for model tuning. We show that the diversity of the machine learning models and careful selection of validation examples are most important for the effectiveness of our approach. Forecasting data have an inherent hierarchical structure (12 levels) but none of our proposed solutions exploited the hierarchical scheme. Using the proposed methodology, we ranked within the gold medal range in the Accuracy track and within the prizes in the Uncertainty track. Inference code with pre-trained models are available on GitHub.¹     

Combination of long term and short term forecasts, with application to tourism demand forecasting

Forecast combination is a well-established and well-tested approach for improving the forecasting accuracy. One beneficial strategy is to use constituent forecasts that have diverse information. In this paper we consider the idea of diversity being accomplished by using different time aggregations. For example, we could create a yearly time series from a monthly time series and produce forecasts for both, then combine the forecasts. These forecasts would each be tracking the dynamics of different time scales, and would therefore add diverse types of information. A comparison of several forecast combination methods, performed in the context of this setup, shows that this is indeed a beneficial strategy and generally provides a forecasting performance that is better than the performances of the individual forecasts that are combined.As a case study, we consider the problem of forecasting monthly tourism numbers for inbound tourism to Egypt. Specifically, we consider 33 individual source countries, as well as the aggregate. The novel combination strategy also produces a generally improved forecasting accuracy.     

Electricity price forecasting: A review of the state-of-the-art with a look into the future

A variety of methods and ideas have been tried for electricity price forecasting (EPF) over the last 15 years, with varying degrees of success. This review article aims to explain the complexity of available solutions, their strengths and weaknesses, and the opportunities and threats that the forecasting tools offer or that may be encountered. The paper also looks ahead and speculates on the directions EPF will or should take in the next decade or so. In particular, it postulates the need for objective comparative EPF studies involving (i) the same datasets, (ii) the same robust error evaluation procedures, and (iii) statistical testing of the significance of one model’s outperformance of another.     

Stability in the inefficient use of forecasting systems: A case study in a supply chain company

Computer-based demand forecasting systems have been widely adopted in supply chain companies, but little research has studied how these systems are actually used in the forecasting process. We report the findings of a case study of demand forecasting in a pharmaceutical company over a 15-year period. At the start of the study, managers believed that they were making extensive use of their forecasting system that was marketed based on the accuracy of its advanced statistical methods. Yet most forecasts were obtained using the system’s facility for judgmentally overriding the automatic statistical forecasts. Carrying out the judgmental interventions involved considerable management effort as part of a sales & operations planning (S&OP) process, yet these often only served to reduce forecast accuracy. This study uses observations of the forecasting process, interviews with participants and data on the accuracy of forecasts to investigate why the managers continued to use non-normative forecasting practices for many years despite the potential economic benefits that could be achieved through change. The reasons for the longevity of these practices are examined both from the perspective of the individual forecaster and the organization as a whole.     

On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks

Daily and weekly seasonalities are always taken into account in day-ahead electricity price forecasting, but the long-term seasonal component has long been believed to add unnecessary complexity, and hence, most studies have ignored it. The recent introduction of the Seasonal Component AutoRegressive (SCAR) modeling framework has changed this viewpoint. However, this framework is based on linear models estimated using ordinary least squares. This paper shows that considering non-linear autoregressive (NARX) neural network-type models with the same inputs as the corresponding SCAR-type models can lead to yet better performances. While individual Seasonal Component Artificial Neural Network (SCANN) models are generally worse than the corresponding SCAR-type structures, we provide empirical evidence that committee machines of SCANN networks can outperform the latter significantly.