A note on amemiya's partially generalized least squares

This note explores the relationship between the generalized least squares estimator and Amemiya's partially generalized least squares estimator and establishes the conditions under which the two estimators are equal.     

Single-equation estimators and aggregation restrictions when equations have the same sets of regressors

It is well known that linear equations subject to cross-equation aggregation restrictions can be ‘stacked’ and estimated simultaneously. However, if every equation contains the same set of regressors, a number of single-equation estimation procedures can be employed. The applicability of ordinary least squares is widely recognized but the article demonstrates that the class of applicable estimators is much broaders than OLS. Under specified conditions, the class includes instrumental variables, generalized least squares, ridge regression, two-stage least squares, k-class estimators, and indirect least squares. Transformations of the original equations and other related matters are discussed also.     

Estimation of limited dependent variable models by ordinary least squares and the method of moments

Under certain conditions, a broad class of qualitative and limited dependent variable models can be consistently estimated by the method of moments using a non-iterative correction to the ordinary least squares estimator, with only a small loss of efficiency compared to maximum likelihood estimation. The class of models is that obtained from a classical multinormal regression by any type of censoring or truncation and includes the tobit, probit, two-limit probit, truncated regression, and some variants of the sample selection models. The paper derives the estimators and their asymptotic covariance matrices.     

Mean estimation bias in least squares estimation of autoregressive processes

Estimation of the parameters of an autoregressive process with a mean that is a function of time is considered. Approximate expressions for the bias of the least squares estimator of the autoregressive parameters that is due to estimating the unknown mean function are derived. For the case of a mean function that is a polynomial in time, a reparameterization that isolates the bias is given. Using the approximate expressions, a method of modifying the least squares estimator is proposed. A Monte Carlo study of the second-order autoregressive process is presented. The Monte Carlo results agree well with the approximate theory and, generally speaking, the modified least squares estimators performed better than the least squares estimator. For the second-order process we also considered the empirical properties of the estimated generalized least squares estimator of the mean function and the error made in predicting the process one, two and three periods in the future.     

A note on least squares estimation and the blue in a generalized linear regression model

In a generalized linear regression model, least squares and Gauss-Markov estimators differ, in general, if the variance-covariance matrix of the disturbances is singular. In the present note it is shown that, nevertheless, the conventional least squares procedure leads to a Gauss-Markov estimator if it is applied to a modified model which results from adding dummy constraints to the original model. These constraints reflect the effects of the singularity of the variance- convariance matrix. As a consequence, a Gauss-Markov estimate may always be obtained by standard least squares minimization, which offers considerable computational advantages.     

More efficient estimation under non-normality when higher moments do not depend on the regressors,using residual augmented least squares

Under normality, least squares is efficient. However, if the errors are not normal, we can gain efficiency from the assertion that higher moments do not depend on the regressors. In this paper, we show how the assumption that higher moments do not depend on the regressors can be exploited in a GMM framework, and we provide simple estimators that are asymptotically equivalent to the GMM estimators. These estimators can be calculated by linear regressions which have been augmented with functions of the least squares residuals.     

On finite sample properties of alternative estimators of coefficients in a structural equation with many instruments

We compare four different estimation methods for the coefficients of a linear structural equation with instrumental variables. As the classical methods we consider the limited information maximum likelihood (LIML) estimator and the two-stage least squares (TSLS) estimator, and as the semi-parametric estimation methods we consider the maximum empirical likelihood (MEL) estimator and the generalized method of moments (GMM) (or the estimating equation) estimator. Tables and figures of the distribution functions of four estimators are given for enough values of the parameters to cover most linear models of interest and we include some heteroscedastic cases and nonlinear cases. We have found that the LIML estimator has good performance in terms of the bounded loss functions and probabilities when the number of instruments is large, that is, the micro-econometric models with “many instruments” in the terminology of recent econometric literature.     

On consistency of the weighted least squares estimators in a semiparametric regression model

This paper is concerned with the semiparametric regression model \(y_i=x_i\beta +g(t_i)+\sigma _ie_i,~~i=1,2,\ldots ,n,\) where \(\sigma _i^2=f(u_i)\), \((x_i,t_i,u_i)\) are known fixed design points, \(\beta \) is an unknown parameter to be estimated, \(g(\cdot )\) and \(f(\cdot )\) are unknown functions, random errors \(e_i\) are widely orthant dependent random variables. The p-th (\(p>0\)) mean consistency and strong consistency for least squares estimators and weighted least squares estimators of \(\beta \) and g under some more mild conditions are investigated. A simulation study is also undertaken to assess the finite sample performance of the results that we established. The results obtained in the paper generalize and improve some corresponding ones of negatively associated random variables.     

Optimality of least squares in the seemingly unrelated regression equation model

In this article the authors have investigated the situations in which the single-equation least squares estimator is identical with the generalized least squares estimator in the seemingly unrelated regression model. The condition obtained turned out to be advantageous from an empirical point of view as it permits one to decide whether to go for a single-equation least squares method or Zellner's method with estimated disturbance variance covariance matrix for estimating the coefficients in the model.     

On the efficiency of least squares estimators in non-linear models

Summary  The identity of least squares estimators å and maximum likelihood estimators â is studied in non-linear models of the type z = g ( a ), where z are observable quantities with a probability density function pr ( z ). This identity was proved for independent random variables z and for distributions pr ( z ), of which the arithmetic sample mean is an optimal estimate.     

Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects

In this paper, I consider generalized least squares (GLS) estimation in fixed effects panel and multilevel models with autocorrelation. The presence of fixed effects complicates implementation of GLS as estimating the fixed effects will typically render standard estimators of the covariance parameters necessary for obtaining feasible GLS estimates inconsistent. I focus on the case where the disturbances follow an AR(p) process and offer a simple to implement bias-correction for the AR coefficients. The usefulness of GLS and the derived bias-correction for the parameters of the autoregressive process is illustrated through a simulation study which uses data from the Current Population Survey.     

Weighted least squares estimators in possibly misspecified nonlinear regression

The behavior of estimators for misspecified parametric models has been well studied. We consider estimators for misspecified nonlinear regression models, with error and covariates possibly dependent. These models are described by specifying a parametric model for the conditional expectation of the response given the covariates. This is a parametric family of conditional constraints, which makes the model itself close to nonparametric. We study the behavior of weighted least squares estimators both when the regression function is correctly specified, and when it is misspecified and also involves possible additional covariates.     

A distribution-free least squares estimator for censored linear regression models

This paper describes a method for estimating simultaneously the parameter vector of the systematic component and the distribution function of the random component of a censored linear regression model. The estimator is obtained by minimizing the sum of the squares of the differences between the observed values of the dependent variable and the corresponding expected values of this variable according to the estimated parameter vector and distribution function. The resulting least squares parameter estimator incorporates information on the distribution of the random component of the regression model that is available from the estimation sample. Hence, it may often be more efficient than are parameter estimators that do not use such information. The results of numerical experiments with the least squares estimator tend to support this hypothesis.     

Statistical inference on seemingly unrelated varying coefficient partially linear models

This paper is concerned with the statistical inference on seemingly unrelated varying coefficient partially linear models. By combining the local polynomial and profile least squares techniques, and estimating the contemporaneous correlation, we propose a class of weighted profile least squares estimators (WPLSEs) for the parametric components. It is shown that the WPLSEs achieve the semiparametric efficiency bound and are asymptotically normal. For the non‐parametric components, by applying the undersmoothing technique, and taking the contemporaneous correlation into account, we propose an efficient local polynomial estimation. The resulting estimators are shown to have mean‐squared errors smaller than those estimators that neglect the contemporaneous correlation. In addition, a class of variable selection procedures is developed for simultaneously selecting significant variables and estimating unknown parameters, based on the non‐concave penalized and weighted profile least squares techniques. With a proper choice of regularization parameters and penalty functions, the proposed variable selection procedures perform as efficiently as if one knew the true submodels. The proposed methods are evaluated using wide simulation studies and applied to a set of real data.     

On the efficiency of least squares estimators in non-linear models

Summary The identity of least squares estimators å and maximum likelihood estimators â is studied in non-linear models of the type z=g(a), where z are observable quantities with a probability density function pr(z). This identity was proved for independent random variables z and for distributions pr(z), of which the arithmetic sample mean is an optimal estimate.     

The asymptotic behaviour of the estimated generalized least squares method in the linear regression model

Abstract  In the linear regression model the generalized least squares (GLS) method is only applicable if the covariance matrix of the errors is known but for a scalar factor. Otherwise an estimator for this matrix has to be used. Then we speak of the estimated generalized least squares (EGLS) method. In this paper the asymptotic behaviour of both methods is compared. Results are applied to some standard models commonly used in econometrics     

Exact inference for the linear model with groupwise heteroscedastic spherical disturbances

Exact nonparametric inference on a single coefficient in a linear regression model, as considered by Bekker (Working Paper, Department of Economics, University of Groningen, 1997), is elaborated for the case of spherically distributed heteroscedastic disturbances. Instead of approximate inference based on feasible weighted least squares, exact inference is formulated based on partial rotational invariance of the distribution of the vector of disturbances. Thus, classical exact inference based on t-statistics is generalized to exact inference that remains valid in a groupwise heteroscedastic context. The approach is applied to a basic two-sample problem, and to the random- and fixed-effects models for panel data.     

Concentration behavior of the penalized least squares estimator

Consider the standard nonparametric regression model and take as estimator the penalized least squares function. In this article, we study the trade‐off between closeness to the true function and complexity penalization of the estimator, where complexity is described by a seminorm on a class of functions. First, we present an exponential concentration inequality revealing the concentration behavior of the trade‐off of the penalized least squares estimator around a nonrandom quantity, where such quantity depends on the problem under consideration. Then, under some conditions and for the proper choice of the tuning parameter, we obtain bounds for this nonrandom quantity. We illustrate our results with some examples that include the smoothing splines estimator.     

On the relative efficiency of estimators which include the initial observations in the estimation of seemingly unrelated regressions with first-order autoregressive disturbances

The generalized least squares estimator for a seemingly unrelated regressions model with first-order vector autoregressive disturbances is outlined, and its efficiency is compared with that of an approximate generalized least squares estimator which ignores the first observation. A scalar index for the loss of efficiency is developed and applied to a special case where the matrix of autoregressive parameters is diagonal and the regressors are smooth. Also, for a more general model, a Monte Carlo study is used to investigate the relative efficiencies of various estimators. The results suggest that Maeshiro (1980) has overstated the case for the exact generalized least squares estimator, because, in many circumstances, it is only marginally better than the approximate generalized least squares estimator.     

Impacts of alternative degrees of freedom corrections in two and three stage least squares

There is a lack of uniformity concerning the appropriate degrees of freedom to use in estimating simultaneous equations. This issue is examined through a Monte Carlo study comparing estimates and inferences obtained using alternative choices of degrees of freedom in two and three stage least squares. While 2SLS estimates do not depend upon this choice, 3SLS estimates do. However, in the study the choice had little impact on 3SLS estimates. But the results strongly suggest that approximate tests conventionally used are much more accurate for both methods if estimated variances account for lost degrees of freedom.