Variable selection via composite quantile regression with dependent errors

We propose composite quantile regression for dependent data, in which the errors are from short‐range dependent and strictly stationary linear processes. Under some regularity conditions, we show that composite quantile estimator enjoys root‐n consistency and asymptotic normality. We investigate the asymptotic relative efficiency of composite quantile estimator to both single‐level quantile regression and least‐squares regression. When the errors have finite variance, the relative efficiency of composite quantile estimator with respect to the least‐squares estimator has a universal lower bound. Under some regularity conditions, the adaptive least absolute shrinkage and selection operator penalty leads to consistent variable selection, and the asymptotic distribution of the non‐zero coefficient is the same as that of the counterparts obtained when the true model is known. We conduct a simulation study and a real data analysis to evaluate the performance of the proposed approach.     

The effect of a Durbin–Watson pretest on confidence intervals in regression

Consider a linear regression model and suppose that our aim is to find a confidence interval for a specified linear combination of the regression parameters. In practice, it is common to perform a Durbin–Watson pretest of the null hypothesis of zero first‐order autocorrelation of the random errors against the alternative hypothesis of positive first‐order autocorrelation. If this null hypothesis is accepted then the confidence interval centered on the ordinary least squares estimator is used; otherwise the confidence interval centered on the feasible generalized least squares estimator is used. For any given design matrix and parameter of interest, we compare the confidence interval resulting from this two‐stage procedure and the confidence interval that is always centered on the feasible generalized least squares estimator, as follows. First, we compare the coverage probability functions of these confidence intervals. Second, we compute the scaled expected length of the confidence interval resulting from the two‐stage procedure, where the scaling is with respect to the expected length of the confidence interval centered on the feasible generalized least squares estimator, with the same minimum coverage probability. These comparisons are used to choose the better confidence interval, prior to any examination of the observed response vector.     

Concentration behavior of the penalized least squares estimator

Consider the standard nonparametric regression model and take as estimator the penalized least squares function. In this article, we study the trade‐off between closeness to the true function and complexity penalization of the estimator, where complexity is described by a seminorm on a class of functions. First, we present an exponential concentration inequality revealing the concentration behavior of the trade‐off of the penalized least squares estimator around a nonrandom quantity, where such quantity depends on the problem under consideration. Then, under some conditions and for the proper choice of the tuning parameter, we obtain bounds for this nonrandom quantity. We illustrate our results with some examples that include the smoothing splines estimator.     

SCAD‐penalized quantile regression for high‐dimensional data analysis and variable selection

The present penalized quantile variable selection methods are only applicable to finite number of predictors or do not have oracle property associated with estimator. This technique is considered as an alternative to ordinary least squares regression in case of the outliers and the heavy‐tailed errors existing in linear models. The variable selection through quantile regression with diverging number of parameters is investigated in this paper. The convergence rate of estimator with smoothly clipped absolute deviation penalty function is also studied. Moreover, the oracle property with proper selection of tuning parameter for quantile regression under certain regularity conditions is also established. In addition, the rank correlation screening method is used to accommodate ultra‐high dimensional data settings. Monte Carlo simulations demonstrate finite performance of the proposed estimator. The results of real data reveal that this approach provides substantially more information as compared with ordinary least squares, conventional quantile regression, and quantile lasso.     

Functional Coefficient Cointegration Models Subject to Time–Varying Volatility with an Application to the Purchasing Power Parity

This paper analyses functional coefficient cointegration models with both stationary and non‐stationary covariates, allowing time‐varying (unconditional) volatility of a general form. The conventional kernel weighted least squares (KLS) estimator is subject to potential efficiency loss, and can be improved by an adaptive kernel weighted least squares (AKLS) estimator that adapts to heteroscedasticity of unknown form. The AKLS estimator is shown to be as efficient as the oracle generalized kernel weighted least squares estimator asymptotically, and can achieve significant efficiency gain relative to the KLS estimator in finite samples. An illustrative example is provided by investigating the Purchasing Power Parity hypothesis.     

Optimality of least squares in the seemingly unrelated regression equation model

In this article the authors have investigated the situations in which the single-equation least squares estimator is identical with the generalized least squares estimator in the seemingly unrelated regression model. The condition obtained turned out to be advantageous from an empirical point of view as it permits one to decide whether to go for a single-equation least squares method or Zellner's method with estimated disturbance variance covariance matrix for estimating the coefficients in the model.     

Estimation in monotone single‐index models

Single‐index models are popular regression models that are more flexible than linear models and still maintain more structure than purely nonparametric models. We consider the problem of estimating the regression parameters under a monotonicity constraint on the unknown link function. In contrast to the standard approach of using smoothing techniques, we review different “non‐smooth” estimators that avoid the difficult smoothing parameter selection. For about 30 years, one has had the conjecture that the profile least squares estimator is an ‐consistent estimator of the regression parameter, but the only non‐smooth argmin/argmax estimators that are actually known to achieve this ‐rate are not based on the nonparametric least squares estimator of the link function. However, solving a score equation corresponding to the least squares approach results in ‐consistent estimators. We illustrate the good behavior of the score approach via simulations. The connection with the binary choice and current status linear regression models is also discussed.     

MSE-improvement of the least squares estimator by dropping variables

It is well known that dropping variables in regression analysis decreases the variance of the least squares (LS) estimator of the remaining parameters. However, after elimination estimates of these parameters are biased, if the full model is correct. In his recent paper, Boscher (1991) showed that the LS-estimator in the special case of a mean shift model (cf. Cook and Weisberg, 1982) which assumes no “outliers” can be considered in the framework of a linear regression model where some variables are deleted. He derived conditions under which this estimator outperforms the LS-estimator of the full model in terms of the mean squared error (MSE)-matrix criterion. We demonstrate that this approach can be extended to the general set-up of dropping variables. Necessary and sufficient conditions for the MSE-matrix superiority of the LS-estimator in the reduced model over that in the full model are derived. We also provide a uniformly most powerful F-statistic for testing the MSE-improvement.     

Another look at the naive estimator in a regression model

We consider the linear regression model where only a particular linear function of the dependent variables is observed, Stahlecker and Schmidt (1987) proposed a naive least squares (LS) estimator for regression coefficients in such a case. In this note we represent their estimator as a general ridge estimator. This observation leads to a view different from the previous work and provides an easy way of obtaining many important properties of the naive LS estimator. Our approach also gives some insight into the relationship between the naive LS estimator and the generalized least squares estimator.     

Partially generalized least squares and two-stage least squares estimators

A class of partially generalized least squares estimators and a class of partially generalized two-stage least squares estimators in regression models with heteroscedastic errors are proposed. By using these estimators a researcher can attain higher efficiency than that attained by the least squares or the two-stage least squares estimators without explicitly estimating each component of the heteroscedastic variances. However, the efficiency is not as high as that of the generalized least squares or the generalized two-stage least squares estimator calculated using the knowledge of the true variances. Hence the use of the term partial.     

Small sample properties of the mixed regression estimator

In this paper, the small sample properties of the mixed regression estimator are examined when prior information may be biased and when the ration of the variance of the prior restriction errors to the variance of the sample errors is unknown. The mean square error of the mixed regression estimator is derived, and it is shown that the mixed regression estimator gets dominated by the ordinary least squares estimator in terms of the mean square error as the bias of prior information gets larger.     

An unexpected property of minimax estimation in the relative squared error approach to linear regression analysis

An unexpected property of the relative squared error approach to linear regression analysis is derived: It is shown that an estimator being minimax among all linear affine estimators is also minimax in the set of all estimators. Two illustrative special cases are mentioned, where a generalized least squares estimator and a general ridge or Kuks-Olman estimator turn out to be minimax.     

On solving bias‐corrected non‐linear estimation equations with an application to the dynamic linear model

In a seminal paper, Mak, Journal of the Royal Statistical Society B, 55, 1993, 945, derived an efficient algorithm for solving non‐linear unbiased estimation equations. In this paper, we show that when Mak's algorithm is applied to biased estimation equations, it results in the estimates that would come from solving a bias‐corrected estimation equation, making it a consistent estimator if regularity conditions hold. In addition, the properties that Mak established for his algorithm also apply in the case of biased estimation equations but for estimates from the bias‐corrected equations. The marginal likelihood estimator is obtained when the approach is applied to both maximum likelihood and least squares estimation of the covariance matrix parameters in the general linear regression model. The new approach results in two new estimators when applied to the profile and marginal likelihood functions for estimating the lagged dependent variable coefficient in the dynamic linear regression model. Monte Carlo simulation results show the new approach leads to a better estimator when applied to the standard profile likelihood. It is therefore recommended for situations in which standard estimators are known to be biased.     

Nonstationary nonlinear heteroskedasticity in regression

This paper considers the regression with errors having nonstationary nonlinear heteroskedasticity. For both the usual stationary regression and the nonstationary cointegrating regression, we develop the asymptotic theories for the least squares methods in the presence of conditional heterogeneity given as a nonlinear function of an integrated process. In particular, we show that the nonstationarity of volatility in the regression errors may induce spuriousness of the underlying regression, if excessive nonstationary volatility is present in the errors. Mild nonstationary volatilities do not render the underlying regression spurious, but their presence makes the least squares estimator asymptotically biased and inefficient and the usual chi-square test invalid.     

Projection‐Based Inference in Randomised Clinical Trials

Covariate information is often available in randomised clinical trials for each subject prior to treatment assignment and is commonly utilised to make covariate adjustment for baseline characteristics predictive of the outcome in order to increase precision and improve power in the detection of a treatment effect. Motivated by a nonparametric covariance analysis, we study a projection approach to making objective covariate adjustment in randomised clinical trials on the basis of two unbiased estimating functions that decouple the outcome and covariate data. The proposed projection approach extends a weighted least‐squares procedure by projecting one of the estimating functions onto the linear subspace spanned by the other estimating function that is E‐ancillary for the average treatment effect. Compared with the weighted least‐squares method, the projection method allows for objective inference on the average treatment effect by exploiting the treatment specific covariate–outcome associations. The resulting projection‐based estimator of the average treatment effect is asymptotically efficient when the treatment‐specific working regression models are correctly specified and is asymptotically more efficient than other existing competitors when the treatment‐specific working regression models are misspecified. The proposed projection method is illustrated by an analysis of data from an HIV clinical trial. In a simulation study, we show that the proposed projection method compares favourably with its competitors in finite samples.     

A note on least squares estimation and the blue in a generalized linear regression model

In a generalized linear regression model, least squares and Gauss-Markov estimators differ, in general, if the variance-covariance matrix of the disturbances is singular. In the present note it is shown that, nevertheless, the conventional least squares procedure leads to a Gauss-Markov estimator if it is applied to a modified model which results from adding dummy constraints to the original model. These constraints reflect the effects of the singularity of the variance- convariance matrix. As a consequence, a Gauss-Markov estimate may always be obtained by standard least squares minimization, which offers considerable computational advantages.     

The asymptotic behaviour of the estimated generalized least squares method in the linear regression model

Abstract  In the linear regression model the generalized least squares (GLS) method is only applicable if the covariance matrix of the errors is known but for a scalar factor. Otherwise an estimator for this matrix has to be used. Then we speak of the estimated generalized least squares (EGLS) method. In this paper the asymptotic behaviour of both methods is compared. Results are applied to some standard models commonly used in econometrics     

Parametric and Nonparametric Regression in the Presence of Endogenous Control Variables

The aim of this paper is to convey to a wider audience of applied statisticians that nonparametric (matching) estimation methods can be a very convenient tool to overcome problems with endogenous control variables. In empirical research one is often interested in the causal effect of a variable X on some outcome variable Y . With observational data, i.e. in the absence of random assignment, the correlation between X and Y generally does not reflect the treatment effect but is confounded by differences in observed and unobserved characteristics. Econometricians often use two different approaches to overcome this problem of confounding by other characteristics. First, controlling for observed characteristics, often referred to as selection on observables, or instrumental variables regression, usually with additional control variables. Instrumental variables estimation is probably the most important estimator in applied work. In many applications, these control variables are themselves correlated with the error term, making ordinary least squares and two-stage least squares inconsistent. The usual solution is to search for additional instrumental variables for these endogenous control variables, which is often difficult. We argue that nonparametric methods help to reduce the number of instruments needed. In fact, we need only one instrument whereas with conventional approaches one may need two, three or even more instruments for consistency. Nonparametric matching estimators permit     consistent estimation without the need for (additional) instrumental variables and permit arbitrary functional forms and treatment effect heterogeneity.     

Fully modified semiparametric GLS estimation for regressions with nonstationary seasonal regressors

Regression models with seasonally integrated and possibly endogenous regressors and serially correlated regression errors are studied. Spectral decompositions of generalized sums of cross products of regressors and regression errors are used to develop a feasible generalized least squares estimator (FGLSE) which does not require parametric specifications for error processes. Using the FGLSE and following the spirit of “Fully Modified estimation” of Phillips and Hansen (Rev. Econ. Stud. 57 (1990) 99), a fully modified GLSE (FM-GLSE) and inference procedures are constructed. The distribution of the FM-GLSE is shown to be asymptotically a mixed normal distribution which validates standard inference based on the FM-GLSE with normal theory. A Monte-Carlo simulation shows that the FM-GLSE is more efficient than the ordinary least squares estimator (OLSE) in the cases of endogeneity or serial correlation and more efficient than the FM-estimator based on the OLSE in the case of serial correlation.     

Testing Convergence in Income Distribution*

The generalized method of moments (GMM) estimator is often used to test for convergence in income distribution in a dynamic panel set‐up. We argue that though consistent, the GMM estimator utilizes the sample observations inefficiently. We propose a simple ordinary least squares (OLS) estimator with more efficient use of sample information. Our Monte Carlo study shows that the GMM estimator can be very imprecise and severely biased in finite samples. In contrast, the OLS estimator overcomes these shortcomings.