The asymptotic behaviour of the estimated generalized least squares method in the linear regression model

Abstract  In the linear regression model the generalized least squares (GLS) method is only applicable if the covariance matrix of the errors is known but for a scalar factor. Otherwise an estimator for this matrix has to be used. Then we speak of the estimated generalized least squares (EGLS) method. In this paper the asymptotic behaviour of both methods is compared. Results are applied to some standard models commonly used in econometrics     

Concentration behavior of the penalized least squares estimator

Consider the standard nonparametric regression model and take as estimator the penalized least squares function. In this article, we study the trade‐off between closeness to the true function and complexity penalization of the estimator, where complexity is described by a seminorm on a class of functions. First, we present an exponential concentration inequality revealing the concentration behavior of the trade‐off of the penalized least squares estimator around a nonrandom quantity, where such quantity depends on the problem under consideration. Then, under some conditions and for the proper choice of the tuning parameter, we obtain bounds for this nonrandom quantity. We illustrate our results with some examples that include the smoothing splines estimator.     

MSE-improvement of the least squares estimator by dropping variables

It is well known that dropping variables in regression analysis decreases the variance of the least squares (LS) estimator of the remaining parameters. However, after elimination estimates of these parameters are biased, if the full model is correct. In his recent paper, Boscher (1991) showed that the LS-estimator in the special case of a mean shift model (cf. Cook and Weisberg, 1982) which assumes no “outliers” can be considered in the framework of a linear regression model where some variables are deleted. He derived conditions under which this estimator outperforms the LS-estimator of the full model in terms of the mean squared error (MSE)-matrix criterion. We demonstrate that this approach can be extended to the general set-up of dropping variables. Necessary and sufficient conditions for the MSE-matrix superiority of the LS-estimator in the reduced model over that in the full model are derived. We also provide a uniformly most powerful F-statistic for testing the MSE-improvement.     

On the performance of tests by Wu and by Hausman for detecting the ordinary least squares bias problem

We first consider the performance of the Wu (1973) - Hausman (1978) (W-H) specification error test as a test for the existence of ordinary least squares (OLS) bias. We discuss power properties of the test under alternative null hypotheses, one of which has not previously been considered. We next consider how the W-H test performs as an indicator of the extent (rather than the existence) of an OLS bias problem, since this usage of the test seems common in applied studies. Finally Monte Carlo methods are used to evaluate Wu's two-step estimation procedure involving the W-H test as a pretest.     

Mean squared error of the empirical best linear unbiased predictor in an orthogonal finite discrete spectrum linear regression model

The mean squared error (MSE) of the empirical best linear unbiased predictor in an orthogonal finite discrete spectrum linear regression model is derived and a comparison with the MSE of the best linear unbiased predictor in this model is made. It is shown that under weak conditions these two mean square errors are asymptotically the same.     

Consistency of elementwise-weighted total least squares estimator in a multivariate errors-in-variables model <Emphasis Type="Bold">AX=B</Emphasis>

A multivariate measurement error model AXB is considered. The errors in [A,B] are rowwise independent, but within each row the errors may be correlated. Some of the columns are observed without errors, and in addition the error covariance matrices may differ from row to row. The total covariance structure of the errors is supposed to be known up to a scalar factor. The fully weighted total least squares estimator of X is studied, which in the case of normal errors coincides with the maximum likelihood estimator. We give mild conditions for weak and strong consistency of the estimator, when the number of rows in A increases. The results generalize the conditions of Gallo given for a univariate homoscedastic model (where B is a vector), and extend the conditions of Gleser given for the multivariate homoscedastic model. We derive the objective function for the estimator and propose an iteratively reweighted numerical procedure.Acknowledgements.A. Kukush is supported by a postdoctoral research fellowship of the Belgian office for Scientific, Technical and Cultural Affairs, promoting Scientific and Technical Collaboration with Central and Eastern Europe. S. Van Huffel is a full professor with the Katholieke Universiteit Leuven. This paper presents research results of the Belgian Programme on Interuniversity Poles of Attraction (IUAP Phase V-22), initiated by the Belgian State, Prime Ministers Office-Federal Office for Scientific, Technical and Cultural Affairs, of the Concerted Research Action (GOA) projects of the Flemish Government MEFISTO-666 (Mathematical Engineering for Information and Communication Systems Technology), of the IDO/99/03 project (K.U. Leuven) Predictive computer models for medical classification problems using patient data and expert knowledge, of the FWO projects G.0200.00, G.0078.01 and G.0270.02. The scientific responsibility is assumed by its authors. The authors would like to thank Maria Luisa Rastello and Amedeo Premoli for bringing the EW-TLS problem to their attention. The authors are grateful to two anonymous referees for the valuable comments.     

Efficiency of iterated WLS in the linear model with completely unknown heteroskedasticity

Iterated weighted least squares (IWLS) is investigated for estimating the regression coefficients in a linear model with symmetrically distributed errors. The variances of the errors are not specified; it is not assumed that they are unknown functions of the explanatory variables nor that they are given in some parametric way.
IWLS is carried out in a random number of steps, of which the first one is OLS. In each step the error variance at time t is estimated with a weighted sum of m squared residuals in the neighbourhood of t and the coefficients are estimated using WLS. Furthermore an estimate of the co-variance matrix is obtained. If this estimate is minimal in some way the iteration process is stopped.
Asymptotic properties of IWLS are derived for increasing sample size n . Some particular cases show that the asymptotic efficiency can be increased by allowing more than two steps. Even asymptotic efficiency with respect to WLS with the true error variances can be obtained if m is not fixed but tends to infinity with n and if the heteroskedasticity is smooth.