Kaggle forecasting competitions: An overlooked learning opportunity

We review the results of six forecasting competitions based on the online data science platform Kaggle, which have been largely overlooked by the forecasting community. In contrast to the M competitions, the competitions reviewed in this study feature daily and weekly time series with exogenous variables, business hierarchy information, or both. Furthermore, the Kaggle data sets all exhibit higher entropy than the M3 and M4 competitions, and they are intermittent.In this review, we confirm the conclusion of the M4 competition that ensemble models using cross-learning tend to outperform local time series models and that gradient boosted decision trees and neural networks are strong forecast methods. Moreover, we present insights regarding the use of external information and validation strategies, and discuss the impacts of data characteristics on the choice of statistics or machine learning methods. Based on these insights, we construct nine ex-ante hypotheses for the outcome of the M5 competition to allow empirical validation of our findings.     

Forecasting an accumulated series based on partial accumulation II: A new Bayesian method for short series with stable seasonal patterns

The problem of forecasting a time series with only a small amount of data is addressed within a Bayesian framework. The quantity to be predicted is the accumulated value of a positive and continuous variable for which partially accumulated data are available. These conditions appear in a natural way in many situations. A simple model is proposed to describe the relationship between the partial and total values of the variable to be forecasted assuming stable seasonality, which is specified in stochastic terms. Analytical results are obtained for both the point forecast and the entire posterior predictive distribution. The proposed technique does not involve approximations. It allows the use of non-informative priors so that implementation may be automatic. The procedure works well when standard methods cannot be applied due to the reduced number of observations. It also improves on previous results published by the authors. Some real examples are included.     

Predictability in sovereign bond returns using technical trading rules: Do developed and emerging markets differ?

The study examines the predictability of 48 sovereign bond markets based on a strategy of 27,000 technical trading rules. These rules represent four popular trading rule classes, they are: moving average, filtering, support and resistance, and channel breakout rules, with numerous variants in each class. Empirical results show that (i) investing in sovereign bond markets is predictable, based on the buy-sell signals generated by trading rules, with the predictability of the emerging Asian markets being significantly higher than those of the advanced markets; (ii) the predictability is generally higher when the US tightens its monetary policies or undergoes recession or a financial crisis; (iii) two-thirds of sovereign bond markets have a higher predictability when we use a machine learning algorithm to determine the best trading rule strategy; and (iv) the predictability of a sovereign bond market is higher when the economy has a less effective government, lower regulatory quality, lower degree of financial openness, higher political risk, lower income and faster real money growth. Our results suggest that shocks originating from US monetary policy or economic conditions could have a considerable spillover effect on sovereign bond markets, particularly the emerging Asian markets.     

The M4 Competition: Results,findings, conclusion and way forward

The M4 competition is the continuation of three previous competitions started more than 45 years ago whose purpose was to learn how to improve forecasting accuracy, and how such learning can be applied to advance the theory and practice of forecasting. The purpose of M4 was to replicate the results of the previous ones and extend them into three directions: First significantly increase the number of series, second include Machine Learning (ML) forecasting methods, and third evaluate both point forecasts and prediction intervals. The five major findings of the M4 Competitions are: 1. Out Of the 17 most accurate methods, 12 were “combinations” of mostly statistical approaches. 2. The biggest surprise was a “hybrid” approach that utilized both statistical and ML features. This method’s average sMAPE was close to 10% more accurate than the combination benchmark used to compare the submitted methods. 3. The second most accurate method was a combination of seven statistical methods and one ML one, with the weights for the averaging being calculated by a ML algorithm that was trained to minimize the forecasting. 4. The two most accurate methods also achieved an amazing success in specifying the 95% prediction intervals correctly. 5. The six pure ML methods performed poorly, with none of them being more accurate than the combination benchmark and only one being more accurate than Naïve2. This paper presents some initial results of M4, its major findings and a logical conclusion. Finally, it outlines what the authors consider to be the way forward for the field of forecasting.