Forecasting electricity prices using bid data

Market liberalization and the expansion of variable renewable energy sources in power systems have made the dynamics of electricity prices more uncertain, leading them to show high volatility with sudden, unexpected price spikes. Thus, developing more accurate price modeling and forecasting techniques is a challenge for all market participants and regulatory authorities. This paper proposes a forecasting approach based on using auction data to fit supply and demand electricity curves. More specifically, we fit linear (LinX-Model) and logistic (LogX-Model) curves to historical sale and purchase bidding data from the Iberian electricity market to estimate structural parameters from 2015 to 2019. Then we use time series models on structural parameters to predict day-ahead prices. Our results provide a solid framework for forecasting electricity prices by capturing the structural characteristics of markets.     

Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx

We extend neural basis expansion analysis (NBEATS) to incorporate exogenous factors. The resulting method, called NBEATSx, improves on a well-performing deep learning model, extending its capabilities by including exogenous variables and allowing it to integrate multiple sources of useful information. To showcase the utility of the NBEATSx model, we conduct a comprehensive study of its application to electricity price forecasting tasks across a broad range of years and markets. We observe state-of-the-art performance, significantly improving the forecast accuracy by nearly 20% over the original NBEATS model, and by up to 5% over other well-established statistical and machine learning methods specialized for these tasks. Additionally, the proposed neural network has an interpretable configuration that can structurally decompose time series, visualizing the relative impact of trend and seasonal components and revealing the modeled processes’ interactions with exogenous factors. To assist related work, we made the code available in a dedicated repository.     

Short term load forecasting and the effect of temperature at the low voltage level

Short term load forecasts will play a key role in the implementation of smart electricity grids. They are required for optimising a wide range of potential network solutions on the low voltage (LV) grid, including the integration of low carbon technologies (such as photovoltaics) and the utilisation of battery storage devices. Despite the need for accurate LV level load forecasts, much of the literature has focused on the individual household or building level using data from smart meters, or on aggregates of such data. This study provides a detailed analysis of several state-of-the-art methods for both point and probabilistic LV load forecasts. We evaluate the out-of-sample forecast accuracies of these methodologies on 100 real LV feeders, for horizons from one to four days ahead. In addition, we also test the effect of the temperature (both actual and forecast) on the accuracy of load forecasts. We present some important results on the drivers of forecasts accuracy as well as on the empirical comparison of point and probabilistic forecast measures.     

Electricity price forecasting: A review of the state-of-the-art with a look into the future

A variety of methods and ideas have been tried for electricity price forecasting (EPF) over the last 15 years, with varying degrees of success. This review article aims to explain the complexity of available solutions, their strengths and weaknesses, and the opportunities and threats that the forecasting tools offer or that may be encountered. The paper also looks ahead and speculates on the directions EPF will or should take in the next decade or so. In particular, it postulates the need for objective comparative EPF studies involving (i) the same datasets, (ii) the same robust error evaluation procedures, and (iii) statistical testing of the significance of one model’s outperformance of another.     

Assessing the accuracy of electricity production forecasts in developing countries

This study assesses the accuracy of time series econometric methods for forecasting electricity production in developing countries. An analysis of the historical time series for 106 developing countries over the period 1960–2012 demonstrates that econometric forecasts are highly accurate for the majority of these countries. These forecasts have much smaller errors than the predictions of simple heuristic models, which assume that electricity production grows at an exogenous rate or is proportional to the real GDP growth. However, the quality of the forecasts diminishes for the countries and regions, where rapid economic and structural transformation makes it difficult to establish stable historical production trends.     

Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition

In this work we introduce the forecasting model with which we participated in the NN5 forecasting competition (the forecasting of 111 time series representing daily cash withdrawal amounts at ATM machines). The main idea of this model is to utilize the concept of forecast combination, which has proven to be an effective methodology in the forecasting literature. In the proposed system we attempted to follow a principled approach, and make use of some of the guidelines and concepts that are known in the forecasting literature to lead to superior performance. For example, we considered various previous comparison studies and time series competitions as guidance in determining which individual forecasting models to test (for possible inclusion in the forecast combination system). The final model ended up consisting of neural networks, Gaussian process regression, and linear models, combined by simple average. We also paid extra attention to the seasonality aspect, decomposing the seasonality into weekly (which is the strongest one), day of the month, and month of the year seasonality.     

Comparison of intraday probabilistic forecasting of solar irradiance using only endogenous data

Accurate solar forecasts are necessary to improve the integration of solar renewables into the energy grid. In recent years, numerous methods have been developed for predicting the solar irradiance or the output of solar renewables. By definition, a forecast is uncertain. Thus, the models developed predict the mean and the associated uncertainty. Comparisons are therefore necessary and useful for assessing the skill and accuracy of these new methods in the field of solar energy.The aim of this paper is to present a comparison of various models that provide probabilistic forecasts of the solar irradiance within a very strict framework. Indeed, we consider focusing on intraday forecasts, with lead times ranging from 1 to 6 h. The models selected use only endogenous inputs for generating the forecasts. In other words, the only inputs of the models are the past solar irradiance data. In this context, the most common way of generating the forecasts is to combine point forecasting methods with probabilistic approaches in order to provide prediction intervals for the solar irradiance forecasts. For this task, we selected from the literature three point forecasting models (recursive autoregressive and moving average (ARMA), coupled autoregressive and dynamical system (CARDS), and neural network (NN)), and seven methods for assessing the distribution of their error (linear model in quantile regression (LMQR), weighted quantile regression (WQR), quantile regression neural network (QRNN), recursive generalized autoregressive conditional heteroskedasticity (GARCHrls), sieve bootstrap (SB), quantile regression forest (QRF), and gradient boosting decision trees (GBDT)), leading to a comparison of 20 combinations of models.None of the model combinations clearly outperform the others; nevertheless, some trends emerge from the comparison. First, the use of the clear sky index ensures the accuracy of the forecasts. This derived parameter permits time series to be deseasonalized with missing data, and is also a good explanatory variable of the distribution of the forecasting errors. Second, regardless of the point forecasting method used, linear models in quantile regression, weighted quantile regression and gradient boosting decision trees are able to forecast the prediction intervals accurately.     

Daily peak electrical load forecasting with a multi-resolution approach

In the context of smart grids and load balancing, daily peak load forecasting has become a critical activity for stakeholders in the energy industry. An understanding of peak magnitude and timing is paramount for the implementation of smart grid strategies such as peak shaving. The modelling approach proposed in this paper leverages high-resolution and low-resolution information to forecast daily peak demand size and timing. The resulting multi-resolution modelling framework can be adapted to different model classes. The key contributions of this paper are (a) a general and formal introduction to the multi-resolution modelling approach, (b) a discussion of modelling approaches at different resolutions implemented via generalised additive models and neural networks, and (c) experimental results on real data from the UK electricity market. The results confirm that the predictive performance of the proposed modelling approach is competitive with that of low- and high-resolution alternatives.     

Forecasting the Chinese stock market volatility with international market volatilities: The role of regime switching

The purpose of this paper is to investigate the role of regime switching in the prediction of the Chinese stock market volatility with international market volatilities. Our work is based on the heterogeneous autoregressive (HAR) model and we further extend this simple benchmark model by incorporating an individual volatility measure from 27 international stock markets. The in-sample estimation results show that the transition probabilities are significant and the high volatility regime exhibits substantially higher volatility level than the low volatility regime. The out-of-sample forecasting results based on the Diebold-Mariano (DM) test suggest that the regime switching models consistently outperform their original counterparts with respect to not only the HAR and its extended models but also the five used combination approaches. In addition to point accuracy, the regime switching models also exhibit substantially higher directional accuracy. Furthermore, compared to time-varying parameter, Markov regime switching is found to be a more efficient way to process the volatility information in the changing world. Our results are also robust to alternative evaluation methods, various loss functions, alternative volatility estimators, various sample periods, and various settings of Markov regime switching. Finally, we provide an extension of forecasting aggregate market volatility on monthly frequency and observe mixed results.     

Description, modelling and forecasting of data with optimal wavelets

Cascade processes have been used to model many different self-similar systems, as they are able to accurately describe most of their global statistical properties. The so-called optimal wavelet basis allows to achieve a geometrical representation of the cascade process-named microcanonical cascade- that describes the behavior of local quantities and thus it helps to reveal the underlying dynamics of the system. In this context, we study the benefits of using the optimal wavelet in contrast to other wavelets when used to define cascade variables, and we provide an optimality degree estimator that is appropriate to determine the closest-to-optimal wavelet in real data. Particularizing the analysis to stock market series, we show that they can be represented by microcanonical cascades in both the logarithm of the price and the volatility. Also, as a promising application in forecasting, we derive the distribution of the value of next point of the series conditioned to the knowledge of past points and the cascade structure, i.e., the stochastic kernel of the cascade process.     

Comparing the forecasting performances of linear models for electricity prices with high RES penetration

We compare alternative univariate versus multivariate models and frequentist versus Bayesian autoregressive and vector autoregressive specifications for hourly day-ahead electricity prices, both with and without renewable energy sources. The accuracy of point and density forecasts is inspected in four main European markets (Germany, Denmark, Italy, and Spain) characterized by different levels of renewable energy power generation. Our results show that the Bayesian vector autoregressive specifications with exogenous variables dominate other multivariate and univariate specifications in terms of both point forecasting and density forecasting.     

A copula-based time series model for global horizontal irradiation

The increasing importance of solar power for electricity generation leads to increasing demand for probabilistic forecasting of local and aggregated photovoltaic (PV) yields. Based on publicly available irradiation data, this paper uses an indirect modeling approach for hourly medium to long-term local PV yields. We suggest a time series model for global horizontal irradiation that allows for multivariate probabilistic forecasts for arbitrary time horizons. It features several important stylized facts. Sharp time-dependent lower and upper bounds of global horizontal irradiations are estimated. The parameters of the beta distributed marginals of the transformed data are allowed to be time-dependent. A copula-based time series model is introduced for the hourly and daily dependence structure based on simple vine copulas with so-called tail dependence. Evaluation methods based on scoring rules are used to compare the model’s power for multivariate probabilistic forecasting with other models used in the literature showing that our model outperforms other models in many respects.     

A white-boxed ISSM approach to estimate uncertainty distributions of Walmart sales

We present our solution for the M5 Uncertainty competition. Our solution ranked sixth out of 909 submissions across all hierarchical levels and ranked first for prediction at the finest level of granularity (product-store sales, i.e. SKUs). The model combines a multi-stage state-space model and Monte Carlo simulations to generate the forecasting scenarios (trajectories). Observed sales are modelled with negative binomial distributions to represent discrete over-dispersed sales. Seasonal factors are handcrafted and modelled with linear coefficients that are calculated at the store-department level.     

Combination of long term and short term forecasts, with application to tourism demand forecasting

Forecast combination is a well-established and well-tested approach for improving the forecasting accuracy. One beneficial strategy is to use constituent forecasts that have diverse information. In this paper we consider the idea of diversity being accomplished by using different time aggregations. For example, we could create a yearly time series from a monthly time series and produce forecasts for both, then combine the forecasts. These forecasts would each be tracking the dynamics of different time scales, and would therefore add diverse types of information. A comparison of several forecast combination methods, performed in the context of this setup, shows that this is indeed a beneficial strategy and generally provides a forecasting performance that is better than the performances of the individual forecasts that are combined.As a case study, we consider the problem of forecasting monthly tourism numbers for inbound tourism to Egypt. Specifically, we consider 33 individual source countries, as well as the aggregate. The novel combination strategy also produces a generally improved forecasting accuracy.     

Booking horizon forecasting with dynamic updating: A case study of hotel reservation data

A highly accurate demand forecast is fundamental to the success of every revenue management model. As is often required in both practice and theory, we aim to forecast the accumulated booking curve, as well as the number of reservations expected for each day in the booking horizon. To reduce the dimensionality of this problem, we apply singular value decomposition to the historical booking profiles. The forecast of the remaining part of the booking horizon is dynamically adjusted to the earlier observations using the penalized least squares and historical proportion methods. Our proposed updating procedure considers the correlation and dynamics of bookings both within the booking horizon and between successive product instances. The approach is tested on real hotel reservation data and shows a significant improvement in forecast accuracy.