Credit Risk Models – Do They Deliver Their Promises? A Quantitative Assessment

We develop a framework to assess the statistical significance of expected default frequency calculated by credit risk models. This framework is then used to analyse the quality of two commercially available models that have become popular among practitioners: KMV Credit Monitor and RiskCalc from Moody's.
Using a unique database of expected default probability from both vendors, we study both the consistency of the prediction and its timeliness. We introduce the concept of cumulative accuracy profile (CAP) that allows to see in one curve the percentage of defaulting companies captured by the models one year in advance. We also use the Miller's information test to see if the models add information to the S&P rating.
The result of the analysis indicates that these models indeed add relevant information not accounted for by rating alone. Moreover, with respect to rating agencies, the models predict defaults more than ten months in advance on average.
(J.E.L.: C52).