A refined parametric model for short term load forecasting

We present a refined parametric model for forecasting electricity demand which performed particularly well in the recent Global Energy Forecasting Competition (GEFCom 2012). We begin by motivating and presenting a simple parametric model, treating the electricity demand as a function of the temperature and day of the data. We then set out a series of refinements of the model, explaining the rationale for each, and using the competition scores to demonstrate that each successive refinement step increases the accuracy of the model’s predictions. These refinements include combining models from multiple weather stations, removing outliers from the historical data, and special treatments of public holidays.     

GEFCom2012: Electric load forecasting and backcasting with semi-parametric models

We sum up the methodology of the team tololo for the Global Energy Forecasting Competition 2012: Load Forecasting. Our strategy consisted of a temporal multi-scale model that combines three components. The first component was a long term trend estimated by means of non-parametric smoothing. The second was a medium term component describing the sensitivity of the electricity demand to the temperature at each time step. We use a generalized additive model to fit this component, using calendar information as well. Finally, a short term component models local behaviours. As the factors that drive this component are unknown, we use a random forest model to estimate it.     

A hybrid machine learning model for forecasting a billing period’s peak electric load days

Many models have been studied for forecasting the peak electric load, but studies focusing on forecasting peak electric load days for a billing period are scarce. This focus is highly relevant to consumers, as their electricity costs are determined based not only on total consumption, but also on the peak load required during a period. Forecasting these peak days accurately allows demand response actions to be planned and executed efficiently in order to mitigate these peaks and their associated costs. We propose a hybrid model based on ARIMA, logistic regression and artificial neural networks models. This hybrid model evaluates the individual results of these statistical and machine learning models in order to forecast whether a given day will be a peak load day for the billing period. The proposed model predicted 70% (40/57) of actual peak load days accurately and revealed potential savings of approximately USD $80,000 for an American university during a one-year testing period.     

A copula-based time series model for global horizontal irradiation

The increasing importance of solar power for electricity generation leads to increasing demand for probabilistic forecasting of local and aggregated photovoltaic (PV) yields. Based on publicly available irradiation data, this paper uses an indirect modeling approach for hourly medium to long-term local PV yields. We suggest a time series model for global horizontal irradiation that allows for multivariate probabilistic forecasts for arbitrary time horizons. It features several important stylized facts. Sharp time-dependent lower and upper bounds of global horizontal irradiations are estimated. The parameters of the beta distributed marginals of the transformed data are allowed to be time-dependent. A copula-based time series model is introduced for the hourly and daily dependence structure based on simple vine copulas with so-called tail dependence. Evaluation methods based on scoring rules are used to compare the model’s power for multivariate probabilistic forecasting with other models used in the literature showing that our model outperforms other models in many respects.     

GEFCom2012 hierarchical load forecasting: Gradient boosting machines and Gaussian processes

This report discusses methods for forecasting hourly loads of a US utility as part of the load forecasting track of the Global Energy Forecasting Competition 2012 hosted on Kaggle. The methods described (gradient boosting machines and Gaussian processes) are generic machine learning/regression algorithms, and few domain-specific adjustments were made. Despite this, the algorithms were able to produce highly competitive predictions, which can hopefully inspire more refined techniques to compete with state-of-the-art load forecasting methodologies.     

A robust support vector regression model for electric load forecasting

Electric load forecasting is a crucial part of business operations in the energy industry. Various load forecasting methods and techniques have been proposed and tested. With growing concerns about cybersecurity and malicious data manipulations, an emerging topic is to develop robust load forecasting models. In this paper, we propose a robust support vector regression (SVR) model to forecast the electricity demand under data integrity attacks. We first introduce a weight function to calculate the relative importance of each observation in the load history. We then construct a weighted quadratic surface SVR model. Some theoretical properties of the proposed model are derived. Extensive computational experiments are based on the publicly available data from Global Energy Forecasting Competition 2012 and ISO New England. To imitate data integrity attacks, we have deliberately increased or decreased the historical load data. Finally, the computational results demonstrate better accuracy of the proposed robust model over other recently proposed robust models in the load forecasting literature.     

Rethinking weather station selection for electric load forecasting using genetic algorithms

Demand forecasting is and has been for years a topic of great interest in the electricity sector, being the temperature one of its major drivers. Indeed, one of the challenges when modelling the load is to choose the right weather station, or set of stations, for a given load time series. However, only a few research papers have been devoted to this topic. This paper reviews the most relevant methods that were applied during the Global Energy Forecasting Competition of 2014 (GEFCom2014) and presents a new approach to weather station selection, based on Genetic Algorithms (GA), which allows finding the best set of stations for any demand forecasting model, and outperforms the results of existing methods. Furthermore its performance has also been tested using GEFCom2012 data, providing significant error improvements. Finally, the possibility of combining the weather stations selected by the proposed GA using the BFGS algorithm is briefly tested, providing promising results.     

Probabilistic forecasting of industrial electricity load with regime switching behavior

This paper suggests a novel inhomogeneous Markov switching approach for the probabilistic forecasting of industrial companies’ electricity loads, for which the load switches at random times between production and standby regimes. The model that we propose describes the transitions between the regimes using a hidden Markov chain with time-varying transition probabilities that depend on calendar variables. We model the demand during the production regime using an autoregressive moving-average (ARMA) process with seasonal patterns, whereas we use a much simpler model for the standby regime in order to reduce the complexity. The maximum likelihood estimation of the parameters is implemented using a differential evolution algorithm. Using the continuous ranked probability score (CRPS) to evaluate the goodness-of-fit of our model for probabilistic forecasting, it is shown that this model often outperforms classical additive time series models, as well as homogeneous Markov switching models. We also propose a simple procedure for classifying load profiles into those with and without regime-switching behaviors.     

Forecasting with gradient boosted trees: augmentation,tuning, and cross-validation strategies: Winning solution to the M5 Uncertainty competition

Deep neural networks and gradient boosted tree models have swept across the field of machine learning over the past decade, producing across-the-board advances in performance. The ability of these methods to capture feature interactions and nonlinearities makes them exceptionally powerful and, at the same time, prone to overfitting, leakage, and a lack of generalization in domains with target non-stationarity and collinearity, such as time-series forecasting. We offer guidance to address these difficulties and provide a framework that maximizes the chances of predictions that generalize well and deliver state-of-the-art performance. The techniques we offer for cross-validation, augmentation, and parameter tuning have been used to win several major time-series forecasting competitions—including the M5 Forecasting Uncertainty competition and the Kaggle COVID19 Forecasting series—and, with the proper theoretical grounding, constitute the current best practices in time-series forecasting.     

Improving Electricity Market Price Forecasting with Factor Models for the Optimal Generation Bid

In liberalized electricity markets, the electricity generation companies usually manage their production by developing hourly bids that are sent to the day‐ahead market. As the prices at which the energy will be purchased are unknown until the end of the bidding process, forecasting of spot prices has become an essential element in electricity management strategies. In this article, we apply forecasting factor models to the market framework in Spain and Portugal and study their performance. Although their goodness of fit is similar to that of autoregressive integrated moving average models, they are easier to implement. The second part of the paper uses the spot‐price forecasting model to generate inputs for a stochastic programming model, which is then used to determine the company's optimal generation bid. The resulting optimal bidding curves are presented and analyzed in the context of the Iberian day‐ahead electricity market.     

Quantile regression for the qualifying match of GEFCom2017 probabilistic load forecasting

We present a simple quantile regression-based forecasting method that was applied in the probabilistic load forecasting framework of the Global Energy Forecasting Competition 2017 (GEFCom2017). The hourly load data are log transformed and split into a long-term trend component and a remainder term. The key forecasting element is the quantile regression approach for the remainder term, which takes into account both weekly and annual seasonalities, such as their interactions. Temperature information is used only for stabilizing the forecast of the long-term trend component. Information on public holidays is ignored. However, the forecasting method still placed second in the open data track and fourth in the definite data track, which is remarkable given the simplicity of the model. The method also outperforms the Vanilla benchmark consistently.     

Operational solar forecasting for the real-time market

Despite the significant progress made in solar forecasting over the last decade, most of the proposed models cannot be readily used by independent system operators (ISOs). This article proposes an operational solar forecasting algorithm that is closely aligned with the real-time market (RTM) forecasting requirements of the California ISO (CAISO). The algorithm first uses the North American Mesoscale (NAM) forecast system to generate hourly forecasts for a 5-h period that are issued 12 h before the actual operating hour, satisfying the lead-time requirement. Subsequently, the world’s fastest similarity search algorithm is adopted to downscale the hourly forecasts generated by NAM to a 15-min resolution, satisfying the forecast-resolution requirement. The 5-h-ahead forecasts are repeated every hour, following the actual rolling update rate of CAISO. Both deterministic and probabilistic forecasts generated using the proposed algorithm are empirically evaluated over a period of 2 years at 7 locations in 5 climate zones.     

Focus forecasting reconsidered

Focus Forecasting is a popular heuristic methodology for production and inventory control although there has never been a rigorous test of accuracy using real time series. We compare Focus Forecasting to damped-trend, seasonal exponential smoothing using five time series of cookware demand in a production planning application. We also make comparisons using 91 time series from the M-Competition study of forecast accuracy. Exponential smoothing was more accurate in both cases.     

Conformal prediction interval estimation and applications to day-ahead and intraday power markets

In this study, we investigated the application of the conformal prediction (CP) concept in the context of short-term electricity price forecasting. In particular, we determined the most important aspects related to the utility of CP, as well as explaining why this simple but highly effective idea has proved useful in other application areas and why its characteristics make it promising for short-term power applications. We compared the performance of CP with various state-of-the-art electricity price forecasting models, such as quantile regression averaging, in an empirical out-of-sample study of three short-term electricity time series. We combined CP with various underlying point forecast models to demonstrate its versatility and behavior under changing conditions. Our findings suggest that CP yields sharp and reliable prediction intervals in short-term power markets. We also inspected the effects of each of the model components to provide path-based guideline regarding how to find the best CP model for each market.     

Comparing the forecasting performances of linear models for electricity prices with high RES penetration

We compare alternative univariate versus multivariate models and frequentist versus Bayesian autoregressive and vector autoregressive specifications for hourly day-ahead electricity prices, both with and without renewable energy sources. The accuracy of point and density forecasts is inspected in four main European markets (Germany, Denmark, Italy, and Spain) characterized by different levels of renewable energy power generation. Our results show that the Bayesian vector autoregressive specifications with exogenous variables dominate other multivariate and univariate specifications in terms of both point forecasting and density forecasting.     

DEMAND,CUSTOMER BASE‐LINE AND DEMAND RESPONSE IN THE ELECTRICITY MARKET: A SURVEY

The increasing use of demand‐side management, as a tool to reliably meet electricity demands at peak time, has stimulated interest among researchers, consumers and producer organiza‐tions, managers, regulators and policymakers. This research reviews the growing literature on models which are used to study demand, customer base‐line (CBL) and demand response in the electricity market. After characterizing the general demand models, the CBL, based on which the demand response models are studied, is reviewed. Given the experience gained from the review and existing conditions, the study combines an appropriate model for each case for a possible application to the electricity market; moreover, it discusses the implications of the results. In the literature, these aspects are studied independently. The main contribution of this survey is attributed to the treatment of the three issues as sequentially interdependent. The review is expected to enhance the understanding of the demand, CBL and demand response in the electricity market and their relationships. The objective is conducted through a combination of demand and supply side managements in order to reduce demand through different demand response programs during peak times. This enables electricity suppliers to save costly electricity generation and at the same time reduce energy vulnerability.     

Reconciled boosted models for GEFCom2017 hierarchical probabilistic load forecasting

When forecasting time series in a hierarchical configuration, it is necessary to ensure that the forecasts reconcile at all levels. The 2017 Global Energy Forecasting Competition (GEFCom2017) focused on addressing this topic. Quantile forecasts for eight zones and two aggregated zones in New England were required for every hour of a future month. This paper presents a new methodology for forecasting quantiles in a hierarchy which outperforms a commonly-used benchmark model. A simulation-based approach was used to generate demand forecasts. Adjustments were made to each of the demand simulations to ensure that all zonal forecasts reconciled appropriately, and a weighted reconciliation approach was implemented to ensure that the bottom-level zonal forecasts summed correctly to the aggregated zonal forecasts. We show that reconciling in this manner improves the forecast accuracy. A discussion of the results and modelling performances is presented, and brief reviews of hierarchical time series forecasting and gradient boosting are also included.     

Robust recurrent network model for intermittent time-series forecasting

This paper describes a deep-learning-based time-series forecasting method that was ranked third in the accuracy challenge of the M5 competition. We solved the problem using a deep-learning approach based on DeepAR, which is an auto-regressive recurrent network model conditioned on historical inputs. To address the intermittent and irregular characteristics of sales demand, we modified the training procedure of DeepAR; instead of using actual values for the historical inputs, our model uses values sampled from a trained distribution and feeds them to the network as past values. We obtained the final result using an ensemble of multiple models to make a robust and stable prediction. To appropriately select a model for the ensemble, each model was evaluated using the average weighted root mean squared scaled error, calculated for all levels of a wide range of past periods.     

Using weather ensemble predictions in electricity demand forecasting

Weather forecasts are an important input to many electricity demand forecasting models. This study investigates the use of weather ensemble predictions in electricity demand forecasting for lead times from 1 to 10 days ahead. A weather ensemble prediction consists of 51 scenarios for a weather variable. We use these scenarios to produce 51 scenarios for the weather-related component of electricity demand. The results show that the average of the demand scenarios is a more accurate demand forecast than that produced using traditional weather forecasts. We use the distribution of the demand scenarios to estimate the demand forecast uncertainty. This compares favourably with estimates produced using univariate volatility forecasting methods.     

Forecasting high resolution electricity demand data with additive models including smooth and jagged components

Short-Term Load Forecasting (STLF) is a fundamental instrument in the efficient operational management and planning of electric utilities. Emerging smart grid technologies pose new challenges and opportunities. Although load forecasting at the aggregate level has been extensively studied, electrical load forecasting at fine-grained geographical scales of households is more challenging. Among existing approaches, semi-parametric generalized additive models (GAM) have been increasingly popular due to their accuracy, flexibility, and interpretability. Their applicability is justified when forecasting is addressed at higher levels of aggregation, since the aggregated load pattern contains relatively smooth additive components. High resolution data are highly volatile, forecasting the average load using GAM models with smooth components does not provide meaningful information about the future demand. Instead, we need to incorporate irregular and volatile effects to enhance the forecast accuracy. We focus on the analysis of such hybrid additive models applied on smart meters data and show that it leads to improvement of the forecasting performances of classical additive models at low aggregation levels.