The asymptotic behaviour of the estimated generalized least squares method in the linear regression model

Abstract  In the linear regression model the generalized least squares (GLS) method is only applicable if the covariance matrix of the errors is known but for a scalar factor. Otherwise an estimator for this matrix has to be used. Then we speak of the estimated generalized least squares (EGLS) method. In this paper the asymptotic behaviour of both methods is compared. Results are applied to some standard models commonly used in econometrics     

Finite sample properties of the QMLE for the Log-ACD model: Application to Australian stocks

This paper concerns the properties of the Quasi Maximum Likelihood Estimator (QMLE) of the Logarithmic Autoregressive Conditional Duration (Log-ACD) model. Proofs of consistency and asymptotic normality of QMLE for the Log-ACD model with log-normal density are presented. This is an important issue as the Log-ACD is used widely for testing various market microstructure models and effects. Knowledge of the distribution of the QMLE is crucial for purposes of valid inference and diagnostic checking. The theoretical results developed in the paper are evaluated using Monte Carlo experiments. The experimental results also provide insights into the finite sample properties of the Log-ACD model under different distributional assumptions. Finally, this paper presents two extensions to the Log-ACD model to accommodate asymmetric effects. The usefulness of these novel models will be evaluated empirically using data from Australian stocks.     

A consistent estimator for nonlinear regression models

In this paper an estimator for the general (nonlinear) regression model with random regressors is studied which is based on the Fourier transform of a certain weight function. Consistency and asymptotic normality of the estimator are established and simulation results are presented to illustrate the theoretical ones.Supported by the Hungarian National Science Foundation OTKA under Grants No. F 032060/2000 and F 046061/2004 and by the Bolyai Grant of the Hungarian Academy of Sciences.Received October 2003     

Semiparametric time series regression modeling with a diverging number of parameters

Variable selection and error structure determination of a partially linear model with time series errors are important issues. In this paper, we investigate the regression coefficient and autoregressive order shrinkage and selection via the smoothly clipped absolute deviation penalty for a partially linear model with a divergent number of covariates and finite order autoregressive time series errors. Both consistency and asymptotic normality of the proposed penalized estimators are derived. The oracle property of the resultant estimators is proved. Simulation studies are carried out to assess the finite‐sample performance of the proposed procedure. A real data analysis is made to illustrate the usefulness of the proposed procedure as well.     

Least absolute deviations estimation for the censored regression model

This paper proposes an alternative to maximum likelihood estimation of the parameters of the censored regression (or censored ‘Tobit’) model. The proposed estimator is a generalization of least absolute deviations estimation for the standard linear model, and, unlike estimation methods based on the assumption of normally distributed error terms, the estimator is consistent and asymptotically normal for a wide class of error distributions, and is also robust to heteroscedasticity. The paper gives the regularity conditions and proofs of these large-sample results, and proposes classes of consistent estimators of the asymptotic covariance matrix for both homoscedastic and heteroscedastic disturbances.     

Semiparametric estimation of a censored regression model with an unknown transformation of the dependent variable

This paper presents a method for estimating the model Λ(Y)=min(β′X+U, C), where Y is a scalar, Λ is an unknown increasing function, X is a vector of explanatory variables, β is a vector of unknown parameters, U has unknown cumulative distribution function F, and C is a censoring threshold. It is not assumed that Λ and F belong to known parametric families; they are estimated nonparametrically. This model includes many widely used models as special cases, including the proportional hazards model with unobserved heterogeneity. The paper develops n^1/2-consistent, asymptotically normal estimators of Λ and F. Estimators of β that are n^1/2-consistent and asymptotically normal already exist. The results of Monte Carlo experiments illustrate the finite-sample behavior of the estimators.     

Threshold estimation for continuous three-phase polynomial regression models with constant mean in the middle regime

This paper considers a continuous three-phase polynomial regression model with two threshold points for dependent data with heteroscedasticity. We assume the model is polynomial of order zero in the middle regime, and is polynomial of higher orders elsewhere. We denote this model by ${ℳ}_2$, which includes models with one or no threshold points, denoted by ${ℳ}_1$ and ${ℳ}_0$, respectively, as special cases. We provide an ordered iterative least squares (OiLS) method when estimating ${ℳ}_2$ and establish the consistency of the OiLS estimators under mild conditions. When the underlying model is ${ℳ}_1$ and is $(d_0-1)$th-order differentiable but not $d_0$th-order differentiable at the threshold point, we further show the $O_p(N^{-1/(d_0+2)})$ convergence rate of the OiLS estimators, which can be faster than the $O_p(N^{-1/(2d_0)})$ convergence rate given in Feder when $d_0\ge 3$. We also apply a model-selection procedure for selecting ${ℳ}_{\kappa }$; $\kappa =0,1,2$. When the underlying model exists, we establish the selection consistency under the aforementioned conditions. Finally, we conduct simulation experiments to demonstrate the finite-sample performance of our asymptotic results.     

Consistent estimation in the bilinear multivariate errors-in-variables model

A bilinear multivariate errors-in-variables model is considered. It corresponds to an overdetermined set of linear equations AXB=C, A∈ℝ^m×n, B∈ℝ^p×q, in which the data A, B, C are perturbed by errors. The total least squares estimator is inconsistent in this case.  An adjusted least squares estimator is constructed, which converges to the true value X, as m →∞, q →∞. A small sample modification of the estimator is presented, which is more stable for small m and q and is asymptotically equivalent to the adjusted least squares estimator. The theoretical results are confirmed by a simulation study. Acknowledgements. We thank two anonymous reviewers for their suggestions and corrections.? A. Kukush is supported by a postdoctoral research fellowship of the Belgian office for Scientific, Technical and Cultural Affairs, promoting Scientific and Technical Collaboration with Central and Eastern Europe.? S. Van Huffel is a full professor with the Katholieke Universiteit Leuven.? I. Markovsky is a research assistant with the Katholieke Universiteit Leuven.? This paper presents research results of the Belgian Programme on Interuniversity Poles of Attraction (IUAP V-22), initiated by the Belgian State, Prime Minister's Office – Federal Office for Scientific, Technical and Cultural Affairs of the Concerted Research Action (GOA) projects of the Flemish Government MEFISTO-666 (Mathematical Engineering for Information and Communication Systems Technology), of the IDO/99/03 project (K.U. Leuven) “Predictive computer models for medical classification problems using patient data and expert knowledge”, of the FWO projects G.0078.01, G.0200.00, and G0.0270.02.? The scientific responsibility is assumed by its authors.     

An application of approximate finite sample results to parameter estimation in a linear errors-in-variables model

In the simple errors-in-variables model the least squares estimator of the slope coefficient is known to be biased towards zero for finite sample size as well as asymptotically. In this paper we suggest a new corrected least squares estimator, where the bias correction is based on approximating the finite sample bias by a lower bound. This estimator is computationally very simple. It is compared with previously proposed corrected least squares estimators, where the correction aims at removing the asymptotic bias or the exact finite sample bias. For each type of corrected least squares estimators we consider the theoretical form, which depends on an unknown parameter, as well as various feasible forms. An analytical comparison of the theoretical estimators is complemented by a Monte Carlo study evaluating the performance of the feasible estimators. The new estimator proposed in this paper proves to be superior with respect to the mean squared error.     

Shrinkage estimation of the exponentiated Weibull regression model for time-to-event data

The exponentiated Weibull distribution is a convenient alternative to the generalized gamma distribution to model time-to-event data. It accommodates both monotone and nonmonotone hazard shapes, and flexible enough to describe data with wide ranging characteristics. It can also be used for regression analysis of time-to-event data. The maximum likelihood method is thus far the most widely used technique for inference, though there is a considerable body of research of improving the maximum likelihood estimators in terms of asymptotic efficiency. For example, there has recently been considerable attention on applying James–Stein shrinkage ideas to parameter estimation in regression models. We propose nonpenalty shrinkage estimation for the exponentiated Weibull regression model for time-to-event data. Comparative studies suggest that the shrinkage estimators outperform the maximum likelihood estimators in terms of statistical efficiency. Overall, the shrinkage method leads to more accurate statistical inference, a fundamental and desirable component of statistical theory.     

Two-step estimation of panel data models with censored endogenous variables and selection bias

This paper presents some two-step estimators for a wide range of parametric panel data models with censored endogenous variables and sample selection bias. Our approach is to derive estimates of the unobserved heterogeneity responsible for the endogeneity/selection bias to include as additional explanatory variables in the primary equation. These are obtained through a decomposition of the reduced form residuals. The panel nature of the data allows adjustment, and testing, for two forms of endogeneity and/or sample selection bias. Furthermore, it incorporates roles for dynamics and state dependence in the reduced form. Finally, we provide an empirical illustration which features our procedure and highlights the ability to test several of the underlying assumptions.     

Asymptotics and smoothing parameter selection for penalized spline regression with various loss functions

Penalized splines are used in various types of regression analyses, including non‐parametric quantile, robust and the usual mean regression. In this paper, we focus on the penalized spline estimator with general convex loss functions. By specifying the loss function, we can obtain the mean estimator, quantile estimator and robust estimator. We will first study the asymptotic properties of penalized splines. Specifically, we will show the asymptotic bias and variance as well as the asymptotic normality of the estimator. Next, we will discuss smoothing parameter selection for the minimization of the mean integrated squares error. The new smoothing parameter can be expressed uniquely using the asymptotic bias and variance of the penalized spline estimator. To validate the new smoothing parameter selection method, we will provide a simulation. The simulation results show that the consistency of the estimator with the proposed smoothing parameter selection method can be confirmed and that the proposed estimator has better behavior than the estimator with generalized approximate cross‐validation. A real data example is also addressed.     

Nonparametric identification and estimation of sample selection models under symmetry

Under a conditional mean restriction Das et al. (2003) considered nonparametric estimation of sample selection models. However, their method can only identify the outcome regression function up to a constant. In this paper we strengthen the conditional mean restriction to a symmetry restriction under which selection biases due to selection on unobservables can be eliminated through proper matching of propensity scores; consequently we are able to identify and obtain consistent estimators for the average treatment effects and the structural regression functions. The results from a simulation study suggest that our estimators perform satisfactorily.     

Finite sample properties of a simple LM test for neglected nonlinearity in error-correcting regression equations

The finite sample properties of LM-type linearity tests based on the discussion in G ranger (1995) are examined. The tests are constructed based on regression models which contain stationary linear and nonlinear functions of nonstationary variables, thus generalizing standard linear cointegrating equations. Power and size simulations are promising, suggesting that the tests are worthy of further examination, and an illustrative empirical example shows that some form of nonlinear error-correction may be useful for explaining the evolution of U.S. money stock in a simple vector autoregression framework.     

A direct Monte Carlo approach for Bayesian analysis of the seemingly unrelated regression model

Computationally efficient methods for Bayesian analysis of seemingly unrelated regression (SUR) models are described and applied that involve the use of a direct Monte Carlo (DMC) approach to calculate Bayesian estimation and prediction results using diffuse or informative priors. This DMC approach is employed to compute Bayesian marginal posterior densities, moments, intervals and other quantities, using data simulated from known models and also using data from an empirical example involving firms’ sales. The results obtained by the DMC approach are compared to those yielded by the use of a Markov Chain Monte Carlo (MCMC) approach. It is concluded from these comparisons that the DMC approach is worthwhile and applicable to many SUR and other problems.     

Efficient estimation of the semiparametric spatial autoregressive model

Efficient semiparametric and parametric estimates are developed for a spatial autoregressive model, containing non-stochastic explanatory variables and innovations suspected to be non-normal. The main stress is on the case of distribution of unknown, nonparametric, form, where series nonparametric estimates of the score function are employed in adaptive estimates of parameters of interest. These estimates are as efficient as the ones based on a correct form, in particular they are more efficient than pseudo-Gaussian maximum likelihood estimates at non-Gaussian distributions. Two different adaptive estimates are considered, relying on somewhat different regularity conditions. A Monte Carlo study of finite sample performance is included.     

A Monte Carlo Study on the Finite Sample Properties of the Gibbs Sampling Method for a Stochastic Frontier Model

In this paper we use Monte Carlo study to investigate the finite sample properties of the Bayesian estimator obtained by the Gibbs sampler and its classical counterpart (i.e. the MLE) for a stochastic frontier model. Our Monte Carlo results show that the MSE performance of the estimates of Gibbs sampling are substantially better than that of the MLE.     

An integrated maximum score estimator for a generalized censored quantile regression model

Quantile regression techniques have been widely used in empirical economics. In this paper, we consider the estimation of a generalized quantile regression model when data are subject to fixed or random censoring. Through a discretization technique, we transform the censored regression model into a sequence of binary choice models and further propose an integrated smoothed maximum score estimator by combining individual binary choice models, following the insights of Horowitz (1992) and Manski (1985). Unlike the estimators of Horowitz (1992) and Manski (1985), our estimators converge at the usual parametric rate through an integration process. In the case of fixed censoring, our approach overcomes a major drawback of existing approaches associated with the curse-of-dimensionality problem. Our approach for the fixed censored case can be extended readily to the case with random censoring for which other existing approaches are no longer applicable. Both of our estimators are consistent and asymptotically normal. A simulation study demonstrates that our estimators perform well in finite samples.     

A Portfolio Index GARCH model

This paper develops the structure of a parsimonious Portfolio Index (PI) GARCH model. Unlike the conventional approach to Portfolio Index returns, which employs the univariate ARCH class, the PI-GARCH approach incorporates the effects on individual assets, leading to a better understanding of portfolio risk management, and achieves greater accuracy in forecasting Value-at-Risk (VaR) thresholds. For various asymmetric GARCH models, a Portfolio Index Composite News Impact Surface (PI-CNIS) is developed to measure the effects of news on the conditional variances. The paper also investigates the finite sample properties of the PI-GARCH model. The empirical example shows that the asymmetric PI-GARCH-t model outperforms the GJR-t model and the filtered historical simulation with a t distribution in forecasting VaR thresholds.     

On consistency of the weighted least squares estimators in a semiparametric regression model

This paper is concerned with the semiparametric regression model \(y_i=x_i\beta +g(t_i)+\sigma _ie_i,~~i=1,2,\ldots ,n,\) where \(\sigma _i^2=f(u_i)\), \((x_i,t_i,u_i)\) are known fixed design points, \(\beta \) is an unknown parameter to be estimated, \(g(\cdot )\) and \(f(\cdot )\) are unknown functions, random errors \(e_i\) are widely orthant dependent random variables. The p-th (\(p>0\)) mean consistency and strong consistency for least squares estimators and weighted least squares estimators of \(\beta \) and g under some more mild conditions are investigated. A simulation study is also undertaken to assess the finite sample performance of the results that we established. The results obtained in the paper generalize and improve some corresponding ones of negatively associated random variables.