Quantification of the high level of endogeneity and of structural regime shifts in commodity markets

We propose a “reflexivity” index that quantifies the relative importance of short-term endogeneity for several commodity futures markets (corn, oil, soybean, sugar, and wheat) and a benchmark equity futures market (E-mini S&P 500), from mid-2000s to October 2012. Our reflexivity index is defined as the average ratio of the number of price moves that are due to endogenous interactions to the total number of all price changes, which also include exogenous events. It is obtained by calibrating the Hawkes self-excited conditional Poisson model on time series of price changes. The Hawkes model accounts simultaneously for the co-existence and interplay between the exogenous impact of news and the endogenous mechanism by which past price changes may influence future price changes. Our robustness tests show that our index provides a ‘pure’ measure of endogeneity that is independent of the rate of activity, order size, volume or volatility. We find an overall increase of the reflexivity index since the mid-2000s to October 2012, which implies that at least 60–70 percent of commodity price changes are now due to self-generated activities rather than novel information, compared to 20–30 percent earlier. While our reflexivity index is defined on short-time windows (10–30 min) and thus does not capture long-term memory, we discover striking coincidence between its dynamics and that of the price hikes and abrupt falls that developed since 2006 and culminated in early 2009.