A tale of forecasting 1001 series : The Bayesian knight strikes again

In this paper it is pointed out that a Bayesian forecasting procedure performed better according to an average mean square error (MSE) criterion than the many other forecasting procedures utilized in the forecasting experiments reported in an extensive study by Makridakis et al. (1982). This fact was not mentioned or discussed by the authors. Also, it is emphasized that if criteria other than MSE are employed, Bayesian forecasts that are optimal relative to them should be employed. Specific examples are provided and analyzed to illustrate this point.     

Adaptive learning forecasting,with applications in forecasting agricultural prices

We introduce a new forecasting methodology, referred to as adaptive learning forecasting, that allows for both forecast averaging and forecast error learning. We analyze its theoretical properties and demonstrate that it provides a priori MSE improvements under certain conditions. The learning rate based on past forecast errors is shown to be non-linear. This methodology is of wide applicability and can provide MSE improvements even for the simplest benchmark models. We illustrate the method’s application using data on agricultural prices for several agricultural products, as well as on real GDP growth for several of the corresponding countries. The time series of agricultural prices are short and show an irregular cyclicality that can be linked to economic performance and productivity, and we consider a variety of forecasting models, both univariate and bivariate, that are linked to output and productivity. Our results support both the efficacy of the new method and the forecastability of agricultural prices.     

Least-squares forecast averaging

This paper proposes forecast combination based on the method of Mallows Model Averaging (MMA). The method selects forecast weights by minimizing a Mallows criterion. This criterion is an asymptotically unbiased estimate of both the in-sample mean-squared error (MSE) and the out-of-sample one-step-ahead mean-squared forecast error (MSFE). Furthermore, the MMA weights are asymptotically mean-square optimal in the absence of time-series dependence. We show how to compute MMA weights in forecasting settings, and investigate the performance of the method in simple but illustrative simulation environments. We find that the MMA forecasts have low MSFE and have much lower maximum regret than other feasible forecasting methods, including equal weighting, BIC selection, weighted BIC, AIC selection, weighted AIC, Bates–Granger combination, predictive least squares, and Granger–Ramanathan combination.     

Mismatch and the Forecasting Performance of Matching Functions

This paper investigates the role of structural imbalance between job seekers and job openings for the forecasting performance of a labour market matching function. Starting from a Cobb–Douglas matching function with constant returns to scale (CRS) in each frictional micro market shows that on the aggregate level, a measure of mismatch is a crucial ingredient of the matching function and hence should not be ignored for forecasting hiring figures. Consequently, we allow the matching process to depend on the level of regional, qualificatory and occupational mismatch between unemployed and vacancies. In pseudo out‐of‐sample tests that account for the nested model environment, we find that forecasting models enhanced by a measure of mismatch significantly outperform their benchmark counterparts for all forecast horizons ranging between one month and a year. This is especially pronounced during and in the aftermath of the Great Recession where a low level of mismatch improved the possibility of unemployed to find a job again. The results show that imposing CRS helps improve forecast accuracy compared to unrestricted models.     

FFORMPP: Feature-based forecast model performance prediction

This paper introduces a novel meta-learning algorithm for time series forecast model performance prediction. We model the forecast error as a function of time series features calculated from historical time series with an efficient Bayesian multivariate surface regression approach. The minimum predicted forecast error is then used to identify an individual model or a combination of models to produce the final forecasts. It is well known that the performance of most meta-learning models depends on the representativeness of the reference dataset used for training. In such circumstances, we augment the reference dataset with a feature-based time series simulation approach, namely GRATIS, to generate a rich and representative time series collection. The proposed framework is tested using the M4 competition data and is compared against commonly used forecasting approaches. Our approach provides comparable performance to other model selection and combination approaches but at a lower computational cost and a higher degree of interpretability, which is important for supporting decisions. We also provide useful insights regarding which forecasting models are expected to work better for particular types of time series, the intrinsic mechanisms of the meta-learners, and how the forecasting performance is affected by various factors.     

Short-term forecasting of crime

The major question investigated is whether it is possible to accurately forecast selected crimes 1 month ahead in small areas, such as police precincts. In a case study of Pittsburgh, PA, we contrast the forecast accuracy of univariate time series models with naïve methods commonly used by police. A major result, expected for the small-scale data of this problem, is that average crime count by precinct is the major determinant of forecast accuracy. A fixed-effects regression model of absolute percent forecast error shows that such counts need to be on the order of 30 or more to achieve accuracy of 20% absolute forecast error or less. A second major result is that practically any model-based forecasting approach is vastly more accurate than current police practices. Holt exponential smoothing with monthly seasonality estimated using city-wide data is the most accurate forecast model for precinct-level crime series.     

Forecasting Errors: the Track Record

This Briefing Paper is the last of a series of three about forecasting. In this one we examine our forecasting record; it complements the February paper in which we analysed the properties of our forecasting model in terms of the error bands attached to the central forecast.
There are many ways of measuring forecasting errors, and in the first part of this Briefing Paper we describe briefing how we have tackled the problem. (A more detailed analysis can be found in the Appendix.) In Part II we report and comment upon the errors in our forecasts of annual growth rates and show how our forecasting performance has improved over the years. In Part III we focus on quarterly forecasts up to 8 quarters ahead, and compare our forecasting errors with measurement errors in the oficial statistics; with the estimation errors built into our forecast equations; and with the stochastic model errors we reported last February. A brief summary of the main conclusions is given below.     

Quantiles as optimal point forecasts

Loss functions play a central role in the theory and practice of forecasting. If the loss function is quadratic, the mean of the predictive distribution is the unique optimal point predictor. If the loss is symmetric piecewise linear, any median is an optimal point forecast. Quantiles arise as optimal point forecasts under a general class of economically relevant loss functions, which nests the asymmetric piecewise linear loss, and which we refer to as generalized piecewise linear (GPL). The level of the quantile depends on a generic asymmetry parameter which reflects the possibly distinct costs of underprediction and overprediction. Conversely, a loss function for which quantiles are optimal point forecasts is necessarily GPL. We review characterizations of this type in the work of Thomson, Saerens and Komunjer, and relate to proper scoring rules, incentive-compatible compensation schemes and quantile regression. In the empirical part of the paper, the relevance of decision theoretic guidance in the transition from a predictive distribution to a point forecast is illustrated using the Bank of England’s density forecasts of United Kingdom inflation rates, and probabilistic predictions of wind energy resources in the Pacific Northwest.     

Forecasting in dynamic models with stochastic regressors

When a dependent variable y is related to present and past values of an exogenous variable x in a dynamic regression (distributed lag) model, and when x must be forecast in order to forecast y, necessary and sufficient conditions are derived in order for optimal forecasts of y to possess lower mean square error as a result of including x in the model, relative to forecasting y solely from its own past. The contribution to this forecast MSE reduction of non-invertibility in the lag distribution is assessed. Examples from econometrics and engineering are provided to illustrate the results.     

Properties of optimal forecasts under asymmetric loss and nonlinearity

Evaluation of forecast optimality in economics and finance has almost exclusively been conducted under the assumption of mean squared error loss. Under this loss function optimal forecasts should be unbiased and forecast errors serially uncorrelated at the single period horizon with increasing variance as the forecast horizon grows. Using analytical results we show that standard properties of optimal forecasts can be invalid under asymmetric loss and nonlinear data generating processes and thus may be very misleading as a benchmark for an optimal forecast. We establish instead that a suitable transformation of the forecast error—known as the generalized forecast error—possesses an equivalent set of properties. The paper also provides empirical examples to illustrate the significance in practice of asymmetric loss and nonlinearities and discusses the effect of parameter estimation error on optimal forecasts.     

Forecasting crude oil price volatility

We use high-frequency intra-day realized volatility data to evaluate the relative forecasting performances of various models that are used commonly for forecasting the volatility of crude oil daily spot returns at multiple horizons. These models include the RiskMetrics, GARCH, asymmetric GARCH, fractional integrated GARCH and Markov switching GARCH models. We begin by implementing Carrasco, Hu, and Ploberger’s (2014) test for regime switching in the mean and variance of the GARCH(1, 1), and find overwhelming support for regime switching. We then perform a comprehensive out-of-sample forecasting performance evaluation using a battery of tests. We find that, under the MSE and QLIKE loss functions: (i) models with a Student’s t innovation are favored over those with a normal innovation; (ii) RiskMetrics and GARCH(1, 1) have good predictive accuracies at short forecast horizons, whereas EGARCH(1, 1) yields the most accurate forecasts at medium horizons; and (iii) the Markov switching GARCH shows a superior predictive accuracy at long horizons. These results are established by computing the equal predictive ability test of Diebold and Mariano (1995) and West (1996) and the model confidence set of Hansen, Lunde, and Nason (2011) over the entire evaluation sample. In addition, a comparison of the MSPE ratios computed using a rolling window suggests that the Markov switching GARCH model is better at predicting the volatility during periods of turmoil.     

组合预测模型在武汉港集装箱吞吐量预测中的应用

为使港口集装箱吞吐量预测的误差更小,精度更高,提出运用弹性系数法、灰色模型法、三次指数平滑法的组合预测模型,预测了武汉港未来特征年的集装箱吞吐量,研究结果表明,组合模型相比单一预测方法能够降低误差、提高精度,预测结果更加理想。     

Calibration of deterministic NWP forecasts and its impact on verification

Deterministic forecasts (as opposed to ensemble or probabilistic forecasts) issued by numerical weather prediction (NWP) models require post-processing. Such corrective procedure can be viewed as a form of calibration. It is well known that, based on different objective functions, e.g., minimizing the mean square error or the mean absolute error, the calibrated forecasts have different impacts on verification. In this regard, this paper investigates how a calibration directive can affect various aspects of forecast quality outlined in the Murphy–Winkler distribution-oriented verification framework. It is argued that the correlation coefficient is the best measure for the potential performance of NWP forecast verification when linear calibration is involved, because (1) it is not affected by the directive of linear calibration, (2) it can be used to compute the skill score of the linearly calibrated forecasts, and (3) it can avoid the potential deficiency of using squared error to rank forecasts. Since no single error metric can fully represent all aspects of forecast quality, forecasters need to understand the trade-offs between different calibration strategies. To echo the increasing need to bridge atmospheric sciences, renewable energy engineering, and power system engineering, as to move toward the grand goal of carbon neutrality, this paper first provides a brief introduction to solar forecasting, and then revolves its discussion around a solar forecasting case study, such that the readers of this journal can gain further understanding on the subject and thus potentially contribute to it.     

Estimating time series models using the relevant cost function

In many forecasting problems, the forecast cost function is used only in evaluating the forecasts; a second cost function is used in estimating the parameters in the model. In this paper, I explore some of the ways in which the forecast cost function can be used in estimating the parameters and, more generally, in producing the forecasts. I define the optimal forecast and note that it may depend on the entire conditional distribution of the data, which is typically unknown. I then consider three of the steps involved in forming the forecast: approximating the optimal forecast, selecting the model, and estimating any unknown parameters. The forecast cost function forms the basis of the approximation, selection, and estimation. The methods are illustrated using time series models applied to 15 US macroeconomic series and in a small Monte Carlo experiment.     

On economic evaluation of directional forecasts

It is commonly accepted that information is helpful if it can be exploited to improve a decision making process. In economics, decisions are often based on forecasts of the upward or downward movements of the variable of interest. We point out that directional forecasts can provide a useful framework for assessing the economic forecast value when loss functions (or success measures) are properly formulated to account for the realized signs and realized magnitudes of directional movements. We discuss a general approach to (directional) forecast evaluation which is based on the loss function proposed by Granger, Pesaran and Skouras. It is simple to implement and provides an economically interpretable loss/success functional framework. We show that, in addition, this loss function is more robust to outlying forecasts than traditional loss functions. As such, the measure of the directional forecast value is a readily available complement to the commonly used squared error loss criterion.     

Performance evaluation of the LDR and the PSH with forecast errors

Despite its ability to produce optimal solutions, the Linear Decision Rule (LDR) has not had a significant impact in the business environment. The Production Switching Heuristic (PSH), which has shown promising results when compared with the LDR, has experienced some business application because of its practicability and flexibility. During aggregate production planning, forecast errors are almost unavoidable, but the sensitivity of these models to such errors has not been thoroughly tested. Insufficient attention has been paid to truly understand the cost effects of forecast errors and other important interactions. The study investigates these issues by analyzing the results of 740 simulated problems.Using the famous “paint factory” cost data, the sensitivity of the LDR and the PSH are examined under various experimental conditions. The factors controlled at different levels are: forecast error mean, forecast error standard deviation, demand pattern, demand variability, and cost coefficients. The results show that 1) the PSH is generally less sensitive than the LDR to forecast errors, 2) both forecast error mean and standard deviation effectively measure the severity of forecast errors, and 3) underforecasts cause less cost penalty than overforecasts.The outcome of the study has helpful managerial implications for aggregate planning related decisionmaking. It suggests that the use of the PSH could result in potential cost savings even if significant forecast errors are envisioned as long as the period-to-period demand variability is not substantially high. Also, BIAS warrants more attention than MSE in evaluating the extent of forecast errors and their eventual cost impact on aggregate production planning.     

Carbon option price forecasting based on modified fractional Brownian motion optimized by GARCH model in carbon emission trading

With the rapid growth of carbon trading, the development of carbon financial derivatives such as carbon options has become inevitable. This paper established a model based on GARCH and fractional Brownian motion (FBM), hoping to provide reference for China's upcoming carbon option trading through carbon option price forecasting research. The fractal characteristic of carbon option prices indicates that it is reasonable to use FBM to predict option prices. The GARCH model can make up for the lack of fixed FBM volatility. In this paper, the daily closing prices of EUA option contracts on the European Energy Exchange are selected as samples for price prediction. The GARCH model was used to determine the return volatility, and then the FBM was used to calculate the forecast price for the next 60 days. The results showed that the predicted price can better fit the actual price. This paper further compares the price prediction results of this model with the other three models through line graphs and error evaluation indicators such as MAPE, MAE and MSE. It is confirmed that the prediction results of the model in this paper is the closest to the actual price.     

Predictive regressions under asymmetric loss: Factor augmentation and model selection

This paper discusses the specifics of forecasting using factor-augmented predictive regressions under general loss functions. In line with the literature, we employ principal component analysis to extract factors from the set of predictors. In addition, we also extract information on the volatility of the series to be predicted, since the volatility is forecast-relevant under non-quadratic loss functions. We ensure asymptotic unbiasedness of the forecasts under the relevant loss by estimating the predictive regression through the minimization of the in-sample average loss. Finally, we select the most promising predictors for the series to be forecast by employing an information criterion that is tailored to the relevant loss. Using a large monthly data set for the US economy, we assess the proposed adjustments in a pseudo out-of-sample forecasting exercise for various variables. As expected, the use of estimation under the relevant loss is found to be effective. Using an additional volatility proxy as the predictor and conducting model selection that is tailored to the relevant loss function enhances the forecast performance significantly.     

Multivariate semi-nonparametric distributions with dynamic conditional correlations

This paper generalizes the Dynamic Conditional Correlation (DCC) model of Engle (2002), incorporating a flexible non-Gaussian distribution based on Gram-Charlier expansions. The resulting semi-nonparametric-DCC (SNP-DCC) model allows estimation in two stages and deals with the negativity problem which is inherent in truncated SNP densities. We test the performance of a SNP-DCC model with respect to the (Gaussian)-DCC through an empirical application of density forecasting for portfolio returns. Our results show that the proposed multivariate model provides a better in-sample fit and forecast of the portfolio returns distribution, and thus is useful for financial risk forecasting and evaluation.     

Residual value forecasting using asymmetric cost functions

Leasing is a popular channel for marketing new cars. However, the pricing of leases is complicated because the leasing rate must embody an expectation of the car’s residual value after contract expiration. This paper develops resale price forecasting models in order to aid pricing decisions. One feature of the leasing business is that different forecast errors entail different costs. The primary objective of this paper is to identify effective ways of addressing cost asymmetry. Specifically, this paper contributes to the literature by (i) consolidating prior work in forecasting on asymmetric functions of the cost of errors; (ii) systematically evaluating previous approaches and comparing them to a new approach; and (iii) demonstrating that forecasting using asymmetric cost of error functions improves the quality of decision support in car leasing. For example, if the costs of overestimating resale prices are twice those of underestimating them, incorporating cost asymmetry into forecast model development reduces costs by about 8%.