Operational solar forecasting for the real-time market

Despite the significant progress made in solar forecasting over the last decade, most of the proposed models cannot be readily used by independent system operators (ISOs). This article proposes an operational solar forecasting algorithm that is closely aligned with the real-time market (RTM) forecasting requirements of the California ISO (CAISO). The algorithm first uses the North American Mesoscale (NAM) forecast system to generate hourly forecasts for a 5-h period that are issued 12 h before the actual operating hour, satisfying the lead-time requirement. Subsequently, the world’s fastest similarity search algorithm is adopted to downscale the hourly forecasts generated by NAM to a 15-min resolution, satisfying the forecast-resolution requirement. The 5-h-ahead forecasts are repeated every hour, following the actual rolling update rate of CAISO. Both deterministic and probabilistic forecasts generated using the proposed algorithm are empirically evaluated over a period of 2 years at 7 locations in 5 climate zones.     

Forecasting Errors: the Track Record

This Briefing Paper is the last of a series of three about forecasting. In this one we examine our forecasting record; it complements the February paper in which we analysed the properties of our forecasting model in terms of the error bands attached to the central forecast.
There are many ways of measuring forecasting errors, and in the first part of this Briefing Paper we describe briefing how we have tackled the problem. (A more detailed analysis can be found in the Appendix.) In Part II we report and comment upon the errors in our forecasts of annual growth rates and show how our forecasting performance has improved over the years. In Part III we focus on quarterly forecasts up to 8 quarters ahead, and compare our forecasting errors with measurement errors in the oficial statistics; with the estimation errors built into our forecast equations; and with the stochastic model errors we reported last February. A brief summary of the main conclusions is given below.     

Forecasts of US short-term interest rates: A flexible forecast combination approach

This paper develops a flexible approach to combine forecasts of future spot rates with forecasts from time-series models or macroeconomic variables. We find empirical evidence that, accounting for both regimes in interest rate dynamics, and combining forecasts from different models, helps improve the out-of-sample forecasting performance for US short-term rates. Imposing restrictions from the expectations hypothesis on the forecasting model are found to help at long forecasting horizons.     

Forecasting with news sentiment: Evidence with UK newspapers

We investigate the performance of newspapers for forecasting inflation, output and unemployment in the United Kingdom. We concentrate on whether the economic policy content reported in popular printed media can improve on existing point forecasts. We find no evidence supporting improved nowcasts or short-term forecasts for inflation. The sentiment inferred from printed media, can however be useful for forecasting unemployment and output. Considerable improvements are also noted when using individual newspapers and keyword based indices.     

A hybrid system-identification method for forecasting telecommunications product demands

A crucial challenge for telecommunications companies is how to forecast changes in demand for specific products over the next 6 to 18 months—the length of a typical short-range capacity-planning and capital-budgeting planning horizon. The problem is especially acute when only short histories of product sales are available. This paper presents a new two-level approach to forecasting demand from short-term data. The lower of the two levels consists of adaptive system-identification algorithms borrowed from signal processing, especially, Hidden Markov Model (HMM) methods [Hidden Markov Models: Estimation and Control (1995) Springer Verlag]. Although they have primarily been used in engineering applications such as automated speech recognition and seismic data processing, HMM techniques also appear to be very promising for predicting probabilities of individual customer behaviors from relatively short samples of recent product-purchasing histories. The upper level of our approach applies a classification tree algorithm to combine information from the lower-level forecasting algorithms. In contrast to other forecast-combination algorithms, such as weighted averaging or Bayesian aggregation formulas, the classification tree approach exploits high-order interactions among error patterns from different predictive systems. It creates a hybrid, forecasting algorithm that out-performs any of the individual algorithms on which it is based. This tree-based approach to hybridizing forecasts provides a new, general way to combine and improve individual forecasts, whether or not they are based on HMM algorithms. The paper concludes with the results of validation tests. These show the power of HMM methods to forecast what individual customers are likely to do next. They also show the gain from classification tree post-processing of the predictions from lower-level forecasts. In essence, these techniques enhance the limited techniques available for new product forecasting.     

Learning,disagreement and inflation forecasting

This paper studies inflation forecasting based on the Bayesian learning algorithm which simultaneously learns about parameters and state variables. The Bayesian learning method updates posterior beliefs with accumulating information from inflation and disagreement about expected inflation from the Survey of Professional Forecasters (SPF). The empirical results show that Bayesian learning helps refine inflation forecasts at all horizons over time. Incorporating a Student’s t innovation improves the accuracy of long-term inflation forecasts. Including disagreement has an effect on refining short-term inflation density forecasts. Furthermore, there is strong evidence supporting a positive correlation between disagreement and trend inflation uncertainty. Our findings are helpful for policymakers when they forecast the future and make forward-looking decisions.     

The role of judgment,assumptions, techniques,and confidence limits in forecasting population

This paper discusses the fundamental role judgment and assumptions play in forecasting population. It is suggested that so-called “projections” operationally are usually either forecasts or extrapolations. Specific projection methodologies and techniques are shown to embody assumptions. A simple typology of such assumptions is presented as a guide to evaluate forecasts. Tests of projection technique accuracy are cited and it is concluded that such tests cannot succeed due to the assumption factor. Finally, time series forecasting techniques are criticized because their terminology with respect to confidence limits about a forecast is misleading.     

Influence of differentiated roles on group forecasting accuracy

While behavioral research on forecasting has mostly examined the individual forecaster, organizationally-based forecasting processes typically tend to rely on groups with members from different functional areas for arriving at ‘consensus’ forecasts. The forecasting performance could also vary depending on the particular group structuring utilized in reaching a final prediction. The current study compares the forecasting performance of modified consensus groups with that of staticized groups using formal role-playing. It is found that, when undistorted model forecasts are given, group forecasts (whether they are arrived at through averaging or by a detailed discussion of the forecasts) contribute positively to the forecasting accuracy. However, providing distorted initial forecasts affects the final accuracy with varying degrees of improvement over the initial forecasts. The results show a strong tendency to favor optimistic forecasts for both the staticized and modified consensus group forecasts. Overall, the role modifications are found to be successful in eliciting a differential adjustment behavior, effectively mimicking the disparities between different organizational roles. Current research suggests that group discussions may be an efficient method of displaying and resolving differential motivational contingencies, potentially leading to group forecasts that perform quite well.     

A large factor model for forecasting macroeconomic variables in South Africa

This paper uses large Factor Models (FMs), which accommodate a large cross-section of macroeconomic time series for forecasting the per capita growth rate, inflation, and the nominal short-term interest rate for the South African economy. The FMs used in this study contain 267 quarterly series observed over the period 1980Q1-2006Q4. The results, based on the RMSEs of one- to four-quarter-ahead out-of-sample forecasts from 2001Q1 to 2006Q4, indicate that the FMs tend to outperform alternative models such as an unrestricted VAR, Bayesian VARs (BVARs) and a typical New Keynesian Dynamic Stochastic General Equilibrium (NKDSGE) model in forecasting the three variables under consideration, hence indicating the blessings of dimensionality.     

Analysts' Forecasts of German Firms' Earnings: a Comparative Analysis

This paper examines analysts' forecasts of the annual earnings per share of German firms over the period of February 1987 to December 1995. The German case is particularly interesting as the accounting and institutional structures vary from those in more thoroughly researched markets such as the U.S. or U.K. The paper therefore considers the features of the German forecasting environment which distinguish it from the Anglo-American model, and whether these might be reflected in forecasting performance. The results for Germany show that the accuracy of analysts' forecasts improves as the forecast horizon shortens, are less accurate than a naive prediction model over longer horizons, and contain a positive bias. When the results for Germany are contrasted with the results for the U.K., as reported in a recent paper, they are found to be a little less accurate but the positive bias is greater in U.K. forecasts. Taken overall the forecasting process in Germany appears to be less efficient than in the U.K., but this may be due to the distinct features of the German forecasting environment.     

Forecasting the volatility of asset returns: The informational gains from option prices

A new class of forecasting models is proposed that extends the realized GARCH class of models through the inclusion of option prices to forecast the variance of asset returns. The VIX is used to approximate option prices, resulting in a set of cross-equation restrictions on the model’s parameters. The full model is characterized by a nonlinear system of three equations containing asset returns, the realized variance, and the VIX, with estimation of the parameters based on maximum likelihood methods. The forecasting properties of the new class of forecasting models, as well as a number of special cases, are investigated and applied to forecasting the daily S&P500 index realized variance using intra-day and daily data from September 2001 to November 2017. The forecasting results provide strong support for including the realized variance and the VIX to improve variance forecasts, with linear conditional variance models performing well for short-term one-day-ahead forecasts, whereas log-linear conditional variance models tend to perform better for intermediate five-day-ahead forecasts.     

Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States

The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden of COVID-19 in the United States from many contributing teams. We study methods for building an ensemble that combines forecasts from these teams. These experiments have informed the ensemble methods used by the Hub. To be most useful to policymakers, ensemble forecasts must have stable performance in the presence of two key characteristics of the component forecasts: (1) occasional misalignment with the reported data, and (2) instability in the relative performance of component forecasters over time. Our results indicate that in the presence of these challenges, an untrained and robust approach to ensembling using an equally weighted median of all component forecasts is a good choice to support public health decision-makers. In settings where some contributing forecasters have a stable record of good performance, trained ensembles that give those forecasters higher weight can also be helpful.     

Stock market volatility forecasting: Do we need high-frequency data?

The general consensus in the volatility forecasting literature is that high-frequency volatility models outperform low-frequency volatility models. However, such a conclusion is reached when low-frequency volatility models are estimated from daily returns. Instead, we study this question considering daily, low-frequency volatility estimators based on open, high, low, and close daily prices. Our data sample consists of 18 stock market indices. We find that high-frequency volatility models tend to outperform low-frequency volatility models only for short-term forecasts. As the forecast horizon increases (up to one month), the difference in forecast accuracy becomes statistically indistinguishable for most market indices. To evaluate the practical implications of our results, we study a simple asset allocation problem. The results reveal that asset allocation based on high-frequency volatility model forecasts does not outperform asset allocation based on low-frequency volatility model forecasts.     

Forecasting Macroeconomic Labour Market Flows: What Can We Learn from Micro‐level Analysis?

Forecasting labour market flows is important for budgeting and decision‐making in government departments and public administration. Macroeconomic forecasts are normally obtained from time series data. In this article, we follow another approach that uses individual‐level statistical analysis to predict the number of exits out of unemployment insurance claims. We present a comparative study of econometric, actuarial and statistical methodologies that base on different data structures. The results with records of the German unemployment insurance suggest that prediction based on individual‐level statistical duration analysis constitutes an interesting alternative to aggregate data‐based forecasting. In particular, forecasts of up to six months ahead are surprisingly precise and are found to be more precise than considered time series forecasts.     

INCOME PERSISTENCE AND MACRO-POLICY FEEDBACKS IN THE US*

This paper analyses and forecasts annual time series of aggregate real income per head in the US. The approach integrates elements from recent univariate time series analyses with multi-equation macromodels in which policy feedback rules have been endogenized. The main conclusions are as follows. Firstly, aggregate real per capita income is subject to significant trend reversion. This conclusion comes through more clearly by examining the data at an annual rather than the more usual quarterly frequency and by incorporating multivariate economic content in the income process. Secondly, there is significant evidence for the Lucas (1976) or Haavelmo (1944) critique: in the US there appears to have been a shift in the structural macropolicy reaction function causing a corresponding shift in the reduced form income forecasting equation. This is associated with increased concern in the late 1980's over the size of US budget deficits. Thirdly, with the above proviso, useful real income forecasts can be made as far as three years ahead. Finally, the paper provides empirical evidence for the effectiveness of monetary policy on real output or income. The change in the short-term interest rate is highly significant in forecasting income growth up to three years after the change.     

Forecasting football match results in national league competitions using score-driven time series models

We develop a new dynamic multivariate model for the analysis and forecasting of football match results in national league competitions. The proposed dynamic model is based on the score of the predictive observation mass function for a high-dimensional panel of weekly match results. Our main interest is in forecasting whether the match result is a win, a loss or a draw for each team. The dynamic model for delivering such forecasts can be based on three different dependent variables: the pairwise count of the number of goals, the difference between the numbers of goals, or the category of the match result (win, loss, draw). The different dependent variables require different distributional assumptions. Furthermore, different dynamic model specifications can be considered for generating the forecasts. We investigate empirically which dependent variable and which dynamic model specification yield the best forecasting results. We validate the precision of the resulting forecasts and the success of the forecasts in a betting simulation in an extensive forecasting study for match results from six large European football competitions. Finally, we conclude that the dynamic model for pairwise counts delivers the most precise forecasts while the dynamic model for the difference between counts is most successful for betting, but that both outperform benchmark and other competing models.     

Long-term forecasting and evaluation

Looking ahead thirty years is a difficult task, but is not impossible. In this paper we illustrate how to evaluate such long-term forecasts. Long-term forecasting is likely to be dominated by trend curves, particularly the simple linear and exponential trends. However, there will certainly be breaks in their parameter values at some unknown points, so that eventually the forecasts will be unsatisfactory. We investigate whether or not simple methods of long-run forecasting can ever be successful, after one takes into account the uncertainty level associated with the forecasts.     

Simple rules for combining forecasts: Some empirical results

This study is an empirical comparison of three rules for aggregating forecasts. The three combined forecasts evaluated are: a simple average forecast, a median forecast and a focus forecast. These combined forecasts are compared over four economic variables (housing starts, the index of industrial production, the unemployment rate and gross national product) using a set of previously published forecasts. The results indicate that an average forecast will not perform as well as previous studies indicate if all or most of the individual forecasts tend to over- or under-predict simultaneously. The median forecast also seems to be suspect in this case. There is little evidence to suggest that the median forecast is a viable alternative to the mean forecast. Focus forecasting, however, is found to perform well for all four variables. The evidence indicates that focus forecasting is a reasonable alternative to simple averaging.     

A disaster response model driven by spatial–temporal forecasts

In this research, we propose a disaster response model combining preparedness and responsiveness strategies. The selective response depends on the level of accuracy that our forecasting models can achieve. In order to decide the right geographical space and time window of response, forecasts are prepared and assessed through a spatial–temporal aggregation framework, until we find the optimum level of aggregation. The research considers major earthquake data for the period 1985–2014. Building on the produced forecasts, we develop accordingly a disaster response model. The model is dynamic in nature, as it is updated every time a new event is added in the database. Any forecasting model can be optimized though the proposed spatial–temporal forecasting framework, and as such our results can be easily generalized. This is true for other forecasting methods and in other disaster response contexts.     

LoMEF: A framework to produce local explanations for global model time series forecasts

Global forecasting models (GFMs) that are trained across a set of multiple time series have shown superior results in many forecasting competitions and real-world applications compared with univariate forecasting approaches. One aspect of the popularity of statistical forecasting models such as ETS and ARIMA is their relative simplicity and interpretability (in terms of relevant lags, trend, seasonality, and other attributes), while GFMs typically lack interpretability, especially relating to particular time series. This reduces the trust and confidence of stakeholders when making decisions based on the forecasts without being able to understand the predictions. To mitigate this problem, we propose a novel local model-agnostic interpretability approach to explain the forecasts from GFMs. We train simpler univariate surrogate models that are considered interpretable (e.g., ETS) on the predictions of the GFM on samples within a neighbourhood that we obtain through bootstrapping, or straightforwardly as the one-step-ahead global black-box model forecasts of the time series which needs to be explained. After, we evaluate the explanations for the forecasts of the global models in both qualitative and quantitative aspects such as accuracy, fidelity, stability, and comprehensibility, and are able to show the benefits of our approach.