A calibration method with dynamic updates for within-match forecasting of wins in tennis

In-match predictions of player win probabilities for professional tennis matches have a wide range of potential applications, including betting, fan engagement, and performance evaluation. The ideal properties of an in-play prediction method include the ability to incorporate both useful pre-match information and relevant in-match information as the match progresses, in order to update the pre-match expectations. This paper presents an in-play forecasting method that achieves both of these goals by combining a pre-match calibration method with a dynamic empirical Bayes updating rule. We present an optimisation rule for guiding the specifications of the dynamic updates using a large sample of professional tennis matches. We apply the results to data from the 2017 season and show that the dynamic model provides a 28% reduction in the error of in-match serve predictions and improves the win prediction accuracy by four percentage points relative to a constant ability model. The method is applied to two Australian Open men’s matches, and we derive several corollary statistics to highlight key dynamics in the win probabilities during a match.     

A profitable model for predicting the over/under market in football

The over/under 2.5 goals betting market allows gamblers to bet on whether the total number of goals in a football match will exceed 2.5. In this paper, a set of ratings, named ‘Generalised Attacking Performance’ (GAP) ratings, are defined which measure the attacking and defensive performance of each team in a league. GAP ratings are used to forecast matches in ten European football leagues and their profitability is tested in the over/under market using two value betting strategies. GAP ratings with match statistics such as shots and shots on target as inputs are shown to yield better predictive value than the number of goals. An average profit of around 0.8 percent per bet taken is demonstrated over twelve years when using only shots and corners (and not goals) as inputs. The betting strategy is shown to be robust by comparing it to a random betting strategy.     

A Bradley-Terry type model for forecasting tennis match results

The paper introduces a model for forecasting match results for the top tier of men’s professional tennis, the ATP tour. Employing a Bradley-Terry type model, and utilising the data available on players’ past results and the surface of the contest, we predict match winners for the coming week’s matches, having updated the model parameters to take the previous week’s results into account. We compare the model to two logit models: one using official rankings and another using the official ranking points of the two competing players. Our model provides superior forecasts according to each of five criteria measuring the predictive performance, two of which relate to betting returns.     

Forecasting football results and exploiting betting markets: The case of “both teams to score”

The continuous growth of available football data presents unprecedented research opportunities for a better understanding of football dynamics. While many research works focus on predicting which team will win a match, other interesting questions, such as whether both teams will score in a game, are still unexplored and have gained momentum with the rise of betting markets. With this in mind, we investigate the following research questions in this paper: “How difficult is the ‘both teams to score’ (BTTS) prediction problem?”, “Are machine learning classifiers capable of predicting BTTS better than bookmakers?”, and “Are machine learning classifiers useful for devising profitable betting strategies in the BTTS market?”. We collected historical football data, extracted groups of features to represent the teams’ strengths, and fed these to state-of-the-art classification models. We performed a comprehensive set of experiments and showed that, although hard to predict, in some scenarios it is possible to outperform bookmakers, which are robust baselines per se. More importantly, in some cases it is possible to beat the market and devise profitable strategies based on machine learning algorithms. The results are encouraging and, besides shedding light on the problem, may provide novel insights for all kinds of football stakeholders.     

Forecasting Singapore's quarterly GDP with monthly external trade

In this paper we suggest a methodology to formulate a dynamic regression with variables observed at different time intervals. This methodology is applicable if the explanatory variables are observed more frequently than the dependent variable. We demonstrate this procedure by developing a forecasting model for Singapore's quarterly GDP based on monthly external trade. Apart from forecasts, the model provides a monthly distributed lag structure between GDP and external trade, which is not possible with quarterly data.     

On determining probability forecasts from betting odds

We show that the probabilities determined from betting odds using Shin’s model are more accurate forecasts than those determined using basic normalization or regression models. We also provide empirical evidence that some bookmakers are significantly different sources of probabilities in terms of forecasting accuracy, and that betting exchange odds are not always the best source, especially in smaller markets. The advantage of using Shin probabilities and the differences between bookmakers decrease with an increasing market size.     

Are betting returns a useful measure of accuracy in (sports) forecasting?

In an economic context, forecasting models are judged in terms not only of accuracy, but also of profitability. The present paper analyses the counterintuitive relationship between accuracy and profitability in probabilistic (sports) forecasts in relation to betting markets. By making use of theoretical considerations, a simulation model, and real-world datasets from three different sports, we demonstrate the possibility of systematically or randomly generating positive betting returns in the absence of a superior model accuracy. The results have methodological implications for sports forecasting and other domains related to betting markets. Betting returns should not be treated as a valid measure of model accuracy, even though they can be regarded as an adequate measure of profitability. Hence, an improved predictive performance might be achieved by carefully considering the roles of both accuracy and profitability when designing models, or, more specifically, when assessing the in-sample fit of data and evaluating out-of-sample forecasting performances.     

The value of statistical forecasts in the UK association football betting market

In this paper, we evaluate the economic significance of statistical forecasts of UK Association Football match outcomes in relation to betting market prices. We present a detailed comparison of odds set by different bookmakers in relation to forecast model predictions, and analyse the potential for arbitrage across firms. We also examine extreme odds biases. A detailed re-examination of match result odds and a new examination of correct score odds for the period 1993 to 1996 suggest that the market is inefficient.     

Forecasting economic time series using score-driven dynamic models with mixed-data sampling

We introduce a mixed-frequency score-driven dynamic model for multiple time series where the score contributions from high-frequency variables are transformed by means of a mixed-data sampling weighting scheme. The resulting dynamic model delivers a flexible and easy-to-implement framework for the forecasting of low-frequency time series variables through the use of timely information from high-frequency variables. We verify the in-sample and out-of-sample performances of the model in an empirical study on the forecasting of U.S. headline inflation and GDP growth. In particular, we forecast monthly headline inflation using daily oil prices and quarterly GDP growth using a measure of financial risk. The forecasting results and other findings are promising. Our proposed score-driven dynamic model with mixed-data sampling weighting outperforms competing models in terms of both point and density forecasts.     

Informational efficiency and behaviour within in-play prediction markets

Studies of financial market informational efficiency have proven burdensome in practice, because it is difficult to pinpoint when news breaks and is known by some or all the participants. We overcome this by designing a framework to detect mispricing, test informational efficiency and evaluate the behavioural biases within high-frequency prediction markets. We demonstrate this using betting exchange data for association football, exploiting the moment when the first goal is scored in a match as major news that breaks cleanly. There are pre-match and in-play mispricing and inefficiency in these markets, explained by reverse favourite-longshot bias (favourite bias). The mispricing tends to increase when the major news is a surprise, such as a goal scored by a longshot team late in a match, with the market underestimating their chances of going on to win These results suggest that, even in prediction markets with large crowds of participants trading state-contingent claims, significant informational inefficiency and behavioural biases can be reflected in prices.     

Forecasting macroeconomic time series: LASSO-based approaches and their forecast combinations with dynamic factor models

In a data-rich environment, forecasting economic variables amounts to extracting and organizing useful information from a large number of predictors. So far, the dynamic factor model and its variants have been the most successful models for such exercises. In this paper, we investigate a category of LASSO-based approaches and evaluate their predictive abilities for forecasting twenty important macroeconomic variables. These alternative models can handle hundreds of data series simultaneously, and extract useful information for forecasting. We also show, both analytically and empirically, that combing forecasts from LASSO-based models with those from dynamic factor models can reduce the mean square forecast error (MSFE) further. Our three main findings can be summarized as follows. First, for most of the variables under investigation, all of the LASSO-based models outperform dynamic factor models in the out-of-sample forecast evaluations. Second, by extracting information and formulating predictors at economically meaningful block levels, the new methods greatly enhance the interpretability of the models. Third, once forecasts from a LASSO-based approach are combined with those from a dynamic factor model by forecast combination techniques, the combined forecasts are significantly better than either dynamic factor model forecasts or the naïve random walk benchmark.     

Leading Indicators for Euro‐area Inflation and GDP Growth*

In this paper, we evaluate the role of a set of variables as leading indicators for Euro‐area inflation and GDP growth. Our leading indicators are taken from the variables in the European Central Bank's (ECB) Euro‐area‐wide model database, plus a set of similar variables for the US. We compare the forecasting performance of each indicator ex post with that of purely autoregressive models. We also analyse three different approaches to combining the information from several indicators. First, ex post, we discuss the use as indicators of the estimated factors from a dynamic factor model for all the indicators. Secondly, within an ex ante framework, an automated model selection procedure is applied to models with a large set of indicators. No future information is used, future values of the regressors are forecast, and the choice of the indicators is based on their past forecasting records. Finally, we consider the forecasting performance of groups of indicators and factors and methods of pooling the ex ante single‐indicator or factor‐based forecasts. Some sensitivity analyses are also undertaken for different forecasting horizons and weighting schemes of forecasts to assess the robustness of the results.     

Forecasts of US short-term interest rates: A flexible forecast combination approach

This paper develops a flexible approach to combine forecasts of future spot rates with forecasts from time-series models or macroeconomic variables. We find empirical evidence that, accounting for both regimes in interest rate dynamics, and combining forecasts from different models, helps improve the out-of-sample forecasting performance for US short-term rates. Imposing restrictions from the expectations hypothesis on the forecasting model are found to help at long forecasting horizons.     

The M5 competition: Background,organization, and implementation

The M5 competition follows the previous four M competitions, whose purpose is to learn from empirical evidence how to improve forecasting performance and advance the theory and practice of forecasting. M5 focused on a retail sales forecasting application with the objective to produce the most accurate point forecasts for 42,840 time series that represent the hierarchical unit sales of the largest retail company in the world, Walmart, as well as to provide the most accurate estimates of the uncertainty of these forecasts. Hence, the competition consisted of two parallel challenges, namely the Accuracy and Uncertainty forecasting competitions. M5 extended the results of the previous M competitions by: (a) significantly expanding the number of participating methods, especially those in the category of machine learning; (b) evaluating the performance of the uncertainty distribution along with point forecast accuracy; (c) including exogenous/explanatory variables in addition to the time series data; (d) using grouped, correlated time series; and (e) focusing on series that display intermittency. This paper describes the background, organization, and implementations of the competition, and it presents the data used and their characteristics. Consequently, it serves as introductory material to the results of the two forecasting challenges to facilitate their understanding.     

The tourism forecasting competition

We evaluate the performances of various methods for forecasting tourism data. The data used include 366 monthly series, 427 quarterly series and 518 annual series, all supplied to us by either tourism bodies or academics who had used them in previous tourism forecasting studies. The forecasting methods implemented in the competition are univariate and multivariate time series approaches, and econometric models. This forecasting competition differs from previous competitions in several ways: (i) we concentrate on tourism data only; (ii) we include approaches with explanatory variables; (iii) we evaluate the forecast interval coverage as well as the point forecast accuracy; (iv) we observe the effect of temporal aggregation on the forecasting accuracy; and (v) we consider the mean absolute scaled error as an alternative forecasting accuracy measure. We find that pure time series approaches provide more accurate forecasts for tourism data than models with explanatory variables. For seasonal data we implement three fully automated pure time series algorithms that generate accurate point forecasts, and two of these also produce forecast coverage probabilities which are satisfactorily close to the nominal rates. For annual data we find that Naïve forecasts are hard to beat.     

The value of feedback in forecasting competitions

In this paper we challenge the traditional design used for forecasting competitions. We implement an online competition with a public leaderboard that provides instant feedback to competitors who are allowed to revise and resubmit forecasts. The results show that feedback significantly improves forecasting accuracy.     

Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting

The Global Energy Forecasting Competition 2017 (GEFCom2017) attracted more than 300 students and professionals from over 30 countries for solving hierarchical probabilistic load forecasting problems. Of the series of global energy forecasting competitions that have been held, GEFCom2017 is the most challenging one to date: the first one to have a qualifying match, the first one to use hierarchical data with more than two levels, the first one to allow the usage of external data sources, the first one to ask for real-time ex-ante forecasts, and the longest one. This paper introduces the qualifying and final matches of GEFCom2017, summarizes the top-ranked methods, publishes the data used in the competition, and presents several reflections on the competition series and a vision for future energy forecasting competitions.     

FRED-SD: A real-time database for state-level data with forecasting applications

We construct a real-time dataset (FRED-SD) with vintage data for the U.S. states that can be used to forecast both state-level and national-level variables. Our dataset includes approximately 28 variables per state, including labor-market, production, and housing variables. We conduct two sets of real-time forecasting exercises. The first forecasts state-level labor-market variables using five different models and different levels of industrially disaggregated data. The second forecasts a national-level variable exploiting the cross-section of state data. The state-forecasting experiments suggest that large models with industrially disaggregated data tend to have higher predictive ability for industrially diversified states. For national-level data, we find that forecasting and aggregating state-level data can outperform a random walk but not an autoregression. We compare these real-time data experiments with forecasting experiments using final-vintage data and find very different results. Because these final-vintage results are obtained with revised data that would not have been available at the time the forecasts would have been made, we conclude that the use of real-time data is essential for drawing proper conclusions about state-level forecasting models.     

You do not have to succeed,just do not fail: When do soccer coaches get fired?

This article deals with circumstances leading to the dismissal of a soccer coach. The article is based on results from the Premier League in England over 12 consecutive years. In this paper, we converted the scores of matches (win, draw, and loss) into an index based upon how results were perceived by club owners—those empowered with the decision as to whether or not to fire the coach. The index is based on the difference between the actual and expected results reflected by betting odds. We conclude that to ensure job preservation, the manager does not have to succeed—he just must not fail.     

Probabilistic recalibration of forecasts

We present a scheme by which a probabilistic forecasting system whose predictions have a poor probabilistic calibration may be recalibrated through the incorporation of past performance information in order to produce a new forecasting system that is demonstrably superior to the original, inasmuch as one may use it to win wagers consistently against someone who is using the original system. The scheme utilizes Gaussian process (GP) modeling to estimate a probability distribution over the probability integral transform (PIT) of a scalar predictand. The GP density estimate gives closed-form access to information entropy measures that are associated with the estimated distribution, which allows the prediction of winnings in wagers against the base forecasting system. A separate consequence of the procedure is that the recalibrated forecast has a uniform expected PIT distribution. One distinguishing feature of the procedure is that it is appropriate even if the PIT values are not i.i.d. The recalibration scheme is formulated in a framework that exploits the deep connections among information theory, forecasting, and betting. We demonstrate the effectiveness of the scheme in two case studies: a laboratory experiment with a nonlinear circuit and seasonal forecasts of the intensity of the El Niño-Southern Oscillation phenomenon.