Data preprocessing and quantile regression for probabilistic load forecasting in the GEFCom2017 final match

Team QUINKAN competed in the GEFCom2017 final match of hierarchical probabilistic load forecasting by adopting the quantile regression method using the R package quantreg. The weather stations were clustered into 11 groups, from which an optimal one was chosen for each load meter using the boosting method. The load meter records were cleaned and/or supplemented by various methods in order to secure robust quantile predictions. The variation in the regression formulas was kept as small as possible by introducing measures for suppressing prediction instability, although special formulas were employed for loading meters that were of an industrial nature. Several procedures were applied to help improve the accuracy, such as the smoothing of season transitions, coarse graining of the relative humidity, the use of load-oriented day-type definition, the averaging of weather data, and outlier removal.     

Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting

The Global Energy Forecasting Competition 2017 (GEFCom2017) attracted more than 300 students and professionals from over 30 countries for solving hierarchical probabilistic load forecasting problems. Of the series of global energy forecasting competitions that have been held, GEFCom2017 is the most challenging one to date: the first one to have a qualifying match, the first one to use hierarchical data with more than two levels, the first one to allow the usage of external data sources, the first one to ask for real-time ex-ante forecasts, and the longest one. This paper introduces the qualifying and final matches of GEFCom2017, summarizes the top-ranked methods, publishes the data used in the competition, and presents several reflections on the competition series and a vision for future energy forecasting competitions.     

Global Energy Forecasting Competition 2012

The Global Energy Forecasting Competition (GEFCom2012) attracted hundreds of participants worldwide, who contributed many novel ideas to the energy forecasting field. This paper introduces both tracks of GEFCom2012, hierarchical load forecasting and wind power forecasting, with details on the aspects of the problem, the data, and a summary of the methods used by selected top entries. We also discuss the lessons learned from this competition from the organizers’ perspective. The complete data set, including the solution data, is published along with this paper, in an effort to establish a benchmark data pool for the community.     

A refined parametric model for short term load forecasting

We present a refined parametric model for forecasting electricity demand which performed particularly well in the recent Global Energy Forecasting Competition (GEFCom 2012). We begin by motivating and presenting a simple parametric model, treating the electricity demand as a function of the temperature and day of the data. We then set out a series of refinements of the model, explaining the rationale for each, and using the competition scores to demonstrate that each successive refinement step increases the accuracy of the model’s predictions. These refinements include combining models from multiple weather stations, removing outliers from the historical data, and special treatments of public holidays.     

Deep learning models for visibility forecasting using climatological data

Low visibility conditions affect safety and traffic operations, leading to adverse scenarios that often result in serious accidents. Due to the complexity and variability associated with modeling weather variables, visibility forecasting remains a highly challenging task and a matter of significant interest for transportation agencies nationwide. Given that the literature on single-step visibility forecasting is very scarce, this study explores the use of deep learning models for single-step visibility forecasting using time series climatological data. Five different deep learning models were developed, trained, and tested using data from two weather stations located in the US state of Florida, which is one of the top states nationwide dealing with low visibility problems. The authors provide discussions of the models’ results and areas for future research.     

一次自动站运行故障解决方法探讨

对一次自动站运行故障解决方法、解决过程中遇到的问题进行阐述,得出地面测报业务人员在日常工作中应注意:按时进行自动站数据、台站参数备份;按时对自动站进行维护;实时对分钟及正点数据检查,加强学习提高自动站维护技能和计算机水平。并为出现同类型故障提供一种可参考的解决方法。     

Temperature anomaly detection for electric load forecasting

Since temperature variables are used in many load forecasting models, the quality of historical temperature data is crucial to the forecast accuracy. The raw data collected by local weather stations and archived by government agencies often include many missing values and incorrect readings, and thus cannot be used directly by load forecasters. As a result, many power companies today purchase data from commercial weather service vendors. Such quality-controlled data may still have many defects, but many load forecasters have been using them in full faith. This paper proposes a novel temperature anomaly detection methodology that makes use of the local load information collected by power companies. The effectiveness of the proposed method is demonstrated through two public datasets: one from the Global Energy Forecasting Competition 2014 and the other from ISO New England. The results show that the accuracy of the final load forecasts can be enhanced by removing the detected observations from the original input data.     

Neural networks for GEFCom2017 probabilistic load forecasting

This report describes the forecasting model which was developed by team “4C” for the global energy forecasting competition 2017 (GEFCom2017), with some modifications added afterwards to improve its accuracy. The model is based on neural networks. Temperature scenarios obtained from historical data are used as inputs to the neural networks in order to create load scenarios, and these load scenarios are then transformed into quantiles. By using a feature selection approach that is based on a stepwise regression technique, a neural network based model is developed for each zone. Furthermore, a dynamic choice of the temperature scenarios is suggested. The feature selection and dynamic choice of the temperature scenarios can improve the quantile scores considerably, resulting in very accurate forecasts among the top teams.     

泵站优化调度控制分析与研究

随着泵站工程的发展,我国泵站的优化调度也逐步深入,而此点是达到整个泵站节能降耗的一种重要方法。文章建立了适合其特点的优化调度数学模型,并对此进行了求解。     

Data visualization and forecast combination for probabilistic load forecasting in GEFCom2017 final match

This paper describes the methods used by Team Cassandra, a joint effort between IBM Research Australia and the University of Melbourne, in the GEFCom2017 load forecasting competition. An important first phase in the forecasting effort involved a deep exploration of the underlying dataset. Several data visualisation techniques were applied to help us better understand the nature and size of gaps, outliers, the relationships between different entities in the dataset, and the relevance of custom date ranges. Improved, cleaned data were then used to train multiple probabilistic forecasting models. These included a number of standard and well-known approaches, as well as a neural-network based quantile forecast model that was developed specifically for this dataset. Finally, model selection and forecast combination were used to choose a custom forecasting model for every entity in the dataset.     

Quantile regression for the qualifying match of GEFCom2017 probabilistic load forecasting

We present a simple quantile regression-based forecasting method that was applied in the probabilistic load forecasting framework of the Global Energy Forecasting Competition 2017 (GEFCom2017). The hourly load data are log transformed and split into a long-term trend component and a remainder term. The key forecasting element is the quantile regression approach for the remainder term, which takes into account both weekly and annual seasonalities, such as their interactions. Temperature information is used only for stabilizing the forecast of the long-term trend component. Information on public holidays is ignored. However, the forecasting method still placed second in the open data track and fourth in the definite data track, which is remarkable given the simplicity of the model. The method also outperforms the Vanilla benchmark consistently.     

区域自动气象观测站报警系统

中尺度自动气象观测站（以下简称自动站）报警系统通过采集分析自动站数据,在地图上直观地显示自动站各气象要素值;通过自动站数据分析自动站的运行状态,当出现自动站运行异常或出现重要天气现象时,以声音、颜色、文字等形式发出警报,以提醒用户注意自动站出现重要情况。同时,中尺度自动气象观测站报警系统提供了自动站数据的二次入库功能,对采集到的自动站数据进行分拆归类,存入标准数据库,以供数据二次开发利用,并在系统中集成了数据查询和基本统计对比功能。     

An ensemble approach to GEFCom2017 probabilistic load forecasting

We present an ensembling approach to medium-term probabilistic load forecasting which ranked second out of 73 competitors in the defined data track of the GEFCom2017 qualifying match. In addition to being accurate, the ensemble method is highly scalable, due to the fact that it had to be applied to nine quantiles in ten zones and for six rounds. Candidate forecasts were generated using random settings for input data, covariates, and learning algorithms. The best candidate forecasts were averaged to create the final forecast, with the number of candidate forecasts being chosen based on their accuracy in similar validation periods.     

A robust support vector regression model for electric load forecasting

Electric load forecasting is a crucial part of business operations in the energy industry. Various load forecasting methods and techniques have been proposed and tested. With growing concerns about cybersecurity and malicious data manipulations, an emerging topic is to develop robust load forecasting models. In this paper, we propose a robust support vector regression (SVR) model to forecast the electricity demand under data integrity attacks. We first introduce a weight function to calculate the relative importance of each observation in the load history. We then construct a weighted quadratic surface SVR model. Some theoretical properties of the proposed model are derived. Extensive computational experiments are based on the publicly available data from Global Energy Forecasting Competition 2012 and ISO New England. To imitate data integrity attacks, we have deliberately increased or decreased the historical load data. Finally, the computational results demonstrate better accuracy of the proposed robust model over other recently proposed robust models in the load forecasting literature.     

浅析DYYZⅡ型自动气象站常见故障及排除方法

通过对自动气象站安装,维修过程中遇到的实际问题的分析,介绍了自动站工作原理、常见故障的判断和处理方法     

中小型水电站网络监控系统的仿真设计

水利水电工程建设周期长、投资大、协作部门多,受自然资源、地形、地质、水文气象条件的影响很大,因而设计一套网络监控系统,对历史水文数据和规划数据进行储存、模拟和分析就显得很有必要。本文首先简单分析了目前水电站网络监控系统的仿真设计现状,然后对中小型水电站网络监控系统的结构及具体设计方案进行探讨与分析,最后就其应用前景进行展望,以供同行参考。     

Reconciled boosted models for GEFCom2017 hierarchical probabilistic load forecasting

When forecasting time series in a hierarchical configuration, it is necessary to ensure that the forecasts reconcile at all levels. The 2017 Global Energy Forecasting Competition (GEFCom2017) focused on addressing this topic. Quantile forecasts for eight zones and two aggregated zones in New England were required for every hour of a future month. This paper presents a new methodology for forecasting quantiles in a hierarchy which outperforms a commonly-used benchmark model. A simulation-based approach was used to generate demand forecasts. Adjustments were made to each of the demand simulations to ensure that all zonal forecasts reconciled appropriately, and a weighted reconciliation approach was implemented to ensure that the bottom-level zonal forecasts summed correctly to the aggregated zonal forecasts. We show that reconciling in this manner improves the forecast accuracy. A discussion of the results and modelling performances is presented, and brief reviews of hierarchical time series forecasting and gradient boosting are also included.     

组合负荷预测方法综述

电力系统负荷具有很多不确定因素,针对单一模型进行负荷预测时,预测精度不高这一问题,可采用组合预测法将多种预测方法所得的预测值进行加权平均而得到最终预测结果,以满足现代电力对负荷预测结果的准确性、快速性和智能化的要求。该文首先简要介绍了几种常用的负荷预测方法,接着详细介绍了组合负荷预测的研究现状及确定组合预测中各模型最优权重的几种方法,最后介绍了组合负荷预测模型的误差修正方法,对提高负荷预测的准确性有一定的现实意义。     

地面最高温度人工站和自动站相差大的成因和处理方法

文章就玉林站地面最高温度值观测中出现的人工站观测与自动站观测的地面最高温度相差较大的成因进行了分析,提出了对基层台站具有实用价值的处理办法,对台站的实际工作有一定的借鉴意义。     

A novel text-based framework for forecasting agricultural futures using massive online news headlines

The agricultural futures prices are generally considered difficult to forecast because the causes of fluctuations are incredibly complicated. We propose a text-based forecasting framework, which can effectively identify and quantify factors affecting agricultural futures based on massive online news headlines. A comprehensive list of influential factors can be formed using a text mining method called topic modeling. A new sentiment-analysis-based way is designed to quantify the factors such as the weather and policies that are important yet difficult to quantify. The proposed framework is empirically tested at forecasting soybean futures prices in the Chinese market. Testing was based on 9715 online news headlines from July 19, 2012 to July 9, 2018. The results show that the identified influential factors and sentiment-based variables are effective, and the proposed framework performs significantly better in medium-term and long-term forecasting than the benchmark model.