Benchmarking regression algorithms for loss given default modeling

The introduction of the Basel II Accord has had a huge impact on financial institutions, allowing them to build credit risk models for three key risk parameters: PD (probability of default), LGD (loss given default) and EAD (exposure at default). Until recently, credit risk research has focused largely on the estimation and validation of the PD parameter, and much less on LGD modeling. In this first large-scale LGD benchmarking study, various regression techniques for modeling and predicting LGD are investigated. These include one-stage models, such as those built by ordinary least squares regression, beta regression, robust regression, ridge regression, regression splines, neural networks, support vector machines and regression trees, as well as two-stage models which combine multiple techniques. A total of 24 techniques are compared using six real-life loss datasets from major international banks. It is found that much of the variance in LGD remains unexplained, as the average prediction performance of the models in terms of R² ranges from 4% to 43%. Nonetheless, there is a clear trend that non-linear techniques, and in particular support vector machines and neural networks, perform significantly better than more traditional linear techniques. Also, two-stage models built by a combination of linear and non-linear techniques are shown to have a similarly good predictive power, with the added advantage of having a comprehensible linear model component.     

Single-equation estimators and aggregation restrictions when equations have the same sets of regressors

It is well known that linear equations subject to cross-equation aggregation restrictions can be ‘stacked’ and estimated simultaneously. However, if every equation contains the same set of regressors, a number of single-equation estimation procedures can be employed. The applicability of ordinary least squares is widely recognized but the article demonstrates that the class of applicable estimators is much broaders than OLS. Under specified conditions, the class includes instrumental variables, generalized least squares, ridge regression, two-stage least squares, k-class estimators, and indirect least squares. Transformations of the original equations and other related matters are discussed also.     

A Geometrical Interpretation of Collinearity: A Natural Way to Justify Ridge Regression and Its Anomalies

Justifying ridge regression from a geometrical perspective is one of the main contributions of this paper. To the best of our knowledge, this question has not been treated previously. This paper shows that ridge regression is a particular case of raising procedures that provide greater flexibility by transforming the matrix X associated with the model. Thus, raising procedures, based on a geometrical idea of the vectorial space associated with the columns of matrix X , lead naturally to ridge regression and justify the presence of the well-known constant k on the main diagonal of matrix X ^′ X . This paper also analyses and compares different alternatives to raising with respect to collinearity mitigation. The results are illustrated with an empirical application.     

Robustness of statistical inferences using linear models with meta-analytic correlation matrices

To examine complex relationships among variables, researchers in human resource management, industrial-organizational psychology, organizational behavior, and related fields have increasingly used meta-analytic procedures to aggregate effect sizes across primary studies to form meta-analytic correlation matrices, which are then subjected to further analyses using linear models (e.g., multiple linear regression). Because missing effect sizes (i.e., correlation coefficients) and different sample sizes across primary studies can occur when constructing meta-analytic correlation matrices, the present study examined the effects of missingness under realistic conditions and various methods for estimating sample size (e.g., minimum sample size, arithmetic mean, harmonic mean, and geometric mean) on the estimated squared multiple correlation coefficient (R²) and the power of the significance test on the overall R² in linear regression. Simulation results suggest that missing data had a more detrimental effect as the number of primary studies decreased and the number of predictor variables increased. It appears that using second-order sample sizes of at least 10 (i.e., independent effect sizes) can improve both statistical power and estimation of the overall R² considerably. Results also suggest that although the minimum sample size should not be used to estimate sample size, the other sample size estimates appear to perform similarly.     

Multi-stage point and interval estimation of the largest mean ofK normal populations and the associated second-order properties

Summary We havek independent normal populations with unknown meansμ ₁, …,μ _k and a common unknown varianceσ ². Both point and interval estimation procedures for the largest mean are proposed by means of sequential and three-stage procedures. For the point estimation problem, we require that the maximal risk be at mostW, a preassigned positive number. For the other problem, we wish to construct a fixed-width confidence interval having the confidence coefficient at least 1-α, a preassigned number between zero and one. Asymptotic second order expansions are provided for various characteristics, such as average sample size, associated risks etc., for the suggested multi-stage estimation procedures.     

The generalized ridge estimator and improved adjustments for regression parameters

Summary The generalized ridge estimator, which considers generalizations of mean square error, is presented, and a mathematical rule of determining the optimalk-value is discussed. The generalized ridge estimator is examined in comparison with the least squares, the pseudoinverse, theJames-Stein-type shrinkage, and the principal component estimators, especially focusing their attention on improved adjustments for regression coefficients. An alternative estimation approach that better integrates a priori information is noted. Finally, combining the generalized ridge and robust regression methods is suggested.     

Shrinkage Estimation Strategies in Generalised Ridge Regression Models: Low/High-Dimension Regime

In this study, we suggest pretest and shrinkage methods based on the generalised ridge regression estimation that is suitable for both multicollinear and high-dimensional problems. We review and develop theoretical results for some of the shrinkage estimators. The relative performance of the shrinkage estimators to some penalty methods is compared and assessed by both simulation and real-data analysis. We show that the suggested methods can be accounted as good competitors to regularisation techniques, by means of a mean squared error of estimation and prediction error. A thorough comparison of pretest and shrinkage estimators based on the maximum likelihood method to the penalty methods. In this paper, we extend the comparison outlined in his work using the least squares method for the generalised ridge regression.     

Standardization of Variables and Collinearity Diagnostic in Ridge Regression

Ridge estimation (RE) is an alternative method to ordinary least squares when there exists a collinearity problem in a linear regression model. The variance inflator factor (VIF) is applied to test if the problem exists in the original model and is also necessary after applying the ridge estimate to check if the chosen value for parameter k has mitigated the collinearity problem. This paper shows that the application of the original data when working with the ridge estimate leads to non‐monotone VIF values. García et al. (2014) showed some problems with the traditional VIF used in RE. We propose an augmented VIF, VIF_R(j,k), associated with RE, which is obtained by standardizing the data before augmenting the model. The VIF_R(j,k) will coincide with the VIF associated with the ordinary least squares estimator when k = 0. The augmented VIF has the very desirable properties of being continuous, monotone in the ridge parameter and higher than one.     

Anomalies in the Foundations of Ridge Regression

Errors persist in ridge regression, its foundations, and its usage, as set forth in Hoerl & Kennard (1970) and elsewhere. Ridge estimators need not be minimizing, nor a prospective ridge parameter be admissible. Conventional estimators are not LaGrange's solutions constrained to fixed lengths, as claimed, since such solutions are singular. Of a massive literature on estimation, prediction, cross–validation, choice of ridge parameter, and related issues, little emanates from constrained optimization to include inequality constraints. The problem traces to a misapplication of LaGrange's Principle, unrecognized singularities, and misplaced links between constraints and ridge parameters. Alternative principles, based on condition numbers, are seen to validate both conventional ridge and surrogate ridge regression to be defined. Numerical studies illustrate that ridge regression as practiced often exhibits pathologies it is intended to redress.     

Instrumental variable quantile regression: A robust inference approach

In this paper, we develop robust inference procedures for an instrumental variables model defined by Y=D^′α(U)$Y=D^{\prime }\alpha (U)$ where D^′α(U)$D^{\prime }\alpha (U)$ is strictly increasing in U and U is a uniform variable that may depend on D but is independent of a set of instrumental variables Z. The proposed inferential procedures are computationally convenient in typical applications and can be carried out using software available for ordinary quantile regression. Our inferential procedure arises naturally from an estimation algorithm and has the important feature of being robust to weak and partial identification and remains valid even in cases where identification fails completely. The use of the proposed procedures is illustrated through two empirical examples.     

The effects of autocorrelation among errors on the consistency property of OLS variance estimator

In a linear regression model the ordinary least squares (OLS) variance estimator (S²) converges in probability to E(S²) even when the errors are autocorrelated. Of interest, however, is the rate of convergence. In this paper we shed some light on this question for the case of a linear trend model. In particular the relation between the rate of convergence and the correlation property of the errors is explored. It is shown that the retardation of the rate of convergence is not appreciable if the correlation is moderate, but it can be severe for extreme correlations.     

Blocked-Error-R 2: A Conceptually Improved Definition of the Proportion of Explained Variance in Models Containing Loops or Correlated Residuals

Bentler and Raykov (2000, Journal of Applied Psychology 85: 125–131), and Jöreskog (1999a, http://www.ssicentral.com/lisrel/column3.htm, 1999b http://www.ssicentral. com/lisrel/column5.htm) proposed procedures for calculating R ² for dependent variables involved in loops or possessing correlated errors. This article demonstrates that Bentler and Raykov’s procedure can not be routinely interpreted as a “proportion” of explained variance, while Jöreskog’s reduced-form calculation is unnecessarily restrictive. The new blocked-error-R ² (beR ²) uses a minimal hypothetical causal intervention to resolve the variance-partitioning ambiguities created by loops and correlated errors. Hayduk (1996) discussed how stabilising feedback models – models capable of counteracting external perturbations – can result in an acceptable error variance which exceeds the variance of the dependent variable to which that error is attached. For variables included within loops, whether stabilising or not, beR ² provides the same value as Hayduk’s (1996) loop-adjusted-R ². For variables not involved in loops and not displaying correlated residuals, beR ² reports the same value as the traditional regression R ². Thus, beR ² provides a conceptualisation of the proportion of explained variance that spans both recursive and nonrecursive structural equation models. A procedure for calculating beR ² in any SEM program is provided.     

Generalised residuals and heterogeneous duration models: With applications to the Weilbull model

This paper deals with models for the duration of an event that are misspecified by the neglect of random multiplicative heterogeneity in the hazard function. This type of misspecification has been widely discussed in the literature [e.g., Heckman and Singer (1982), Lancaster and Nickell (1980)], but no study of its effect on maximum likelihood estimators has been given. This paper aims to provide such a study with particular reference to the Weibull regression model which is by far the most frequently used parametric model [e.g., Heckman and Borjas (1980), Lancaster (1979)]. In this paper we define generalised errors and residuals in the sense of Cox and Snell (1968, 1971) and show how their use materially simplifies the analysis of both true and misspecified duration models. We show that multiplicative heterogeneity in the hazard of the Weibull model has two errors in variables interpretations. We give the exact asymptotic inconsistency of M.L. estimation in the Weibull model and give a general expression for the inconsistency of M.L. estimators due to neglected heterogeneity for any duration model to O(σ²), where σ² is the variance of the error term. We also discuss the information matrix test for neglected heterogeneity in duration models and consider its behaviour when σ²>0.     

Semiparametric Bayesian inference in smooth coefficient models

We describe procedures for Bayesian estimation and testing in cross-sectional, panel data and nonlinear smooth coefficient models. The smooth coefficient model is a generalization of the partially linear or additive model wherein coefficients on linear explanatory variables are treated as unknown functions of an observable covariate. In the approach we describe, points on the regression lines are regarded as unknown parameters and priors are placed on differences between adjacent points to introduce the potential for smoothing the curves. The algorithms we describe are quite simple to implement—for example, estimation, testing and smoothing parameter selection can be carried out analytically in the cross-sectional smooth coefficient model.     

Estimating a tournament model of intra-firm wage differentials

We consider the estimation of a tournament model with moral hazard (based on Rosen (1986), AER)) when only aggregate data on intra-firm employment levels and salaries are available. Equilibrium restrictions of the model allow us to recover parameters of interest, including equilibrium effort levels in each hierarchical stage of the firm. We illustrate our estimation procedures using data from major retail chains in the US. We find that only a fraction of the wage differential directly compensates workers for higher effort levels, implying that a large portion of the differentials arises to maintain incentives at lower rungs of the retailers.     

Identification robust confidence set methods for inference on parameter ratios with application to discrete choice models

We study the problem of building confidence sets for ratios of parameters, from an identification robust perspective. In particular, we address the simultaneous confidence set estimation of a finite number of ratios. Results apply to a wide class of models suitable for estimation by consistent asymptotically normal procedures. Conventional methods (e.g. the delta method) derived by excluding the parameter discontinuity regions entailed by the ratio functions and which typically yield bounded confidence limits, break down even if the sample size is large ( Dufour, 1997). One solution to this problem, which we take in this paper, is to use variants of  Fieller’s ( 1940, 1954) method. By inverting a joint test that does not require identifying the ratios, Fieller-based confidence regions are formed for the full set of ratios. Simultaneous confidence sets for individual ratios are then derived by applying projection techniques, which allow for possibly unbounded outcomes. In this paper, we provide simple explicit closed-form analytical solutions for projection-based simultaneous confidence sets, in the case of linear transformations of ratios. Our solution further provides a formal proof for the expressions in Zerbe et al. (1982) pertaining to individual ratios. We apply the geometry of quadrics as introduced by  and , in a different although related context. The confidence sets so obtained are exact if the inverted test statistic admits a tractable exact distribution, for instance in the normal linear regression context. The proposed procedures are applied and assessed via illustrative Monte Carlo and empirical examples, with a focus on discrete choice models estimated by exact or simulation-based maximum likelihood. Our results underscore the superiority of Fieller-based methods.     

Generalized variance-ratio tests for serial correlation in multivariate regression models

Exact tests for rth order serial correlation in the multivariate linear regression model are devised which are based on a multivariate generalization of the F-distribution. The tests require the computation of two multivariate regressions. In the special case of a single-equation regression model the procedures reduce to simple always-conclusive F-tests. The tests are illustrated by applications to the Rotterdam Model of consumer demand.     

Reduced rank regression in cointegrated models

The coefficient matrix of a cointegrated first-order autoregression is estimated by reduced rank regression (RRR), depending on the larger canonical correlations and vectors of the first difference of the observed series and the lagged variables. In a suitable coordinate system the components of the least-squares (LS) estimator associated with the lagged nonstationary variables are of order 1/T, where T is the sample size, and are asymptotically functionals of a Brownian motion process; the components associated with the lagged stationary variables are of the order T^−1/2 and are asymptotically normal. The components of the RRR estimator associated with the stationary part are asymptotically the same as for the LS estimator. Some components of the RRR estimator associated with nonstationary regressors have zero error to order 1/T and the other components have a more concentrated distribution than the corresponding components of the LS estimator.     

Estimation of variance after a preliminary test of homogeneity and optimal levels of significance for the pre-test

The question of whether to pool two samples in variance estimation is often decided via a preliminary F test. In this paper we show that the optimal pre-test F value is unity for a one- sided alternative, where the objective function is to minimize average relative risk. The outcome is independent of numbers of degrees of freedom in each sample. Optimal significance levels vary somewhat but are close to $\text{1}\text{2}$ for most d.f. and equal to $\text{1}\text{2}$ when numerator and denominator d.f. are equal. The results also apply to regression variance estimation across two data regimes.     

Edgeworth approximations for semiparametric instrumental variable estimators and test statistics

We establish the validity of higher order asymptotic expansions to the distribution of a version of the nonlinear semiparametric instrumental variable estimator considered in Newey (Econometrica 58 (1990) 809) as well as to the distribution of a Wald statistic derived from it. We employ local polynomial smoothing with variable bandwidth, which includes local linear, kernel, and (a version of) nearest neighbor estimates as special cases. Our expansions are valid to order n^−2ε for some 0<ε<1/2, where ε depends on the smoothness and dimensionality of the data distribution and on the order of the polynomial chosen by the practitioner. We use the expansions to define optimal bandwidth selection methods for both estimation and testing problems and apply our methods to simulated data.