SCAD‐penalized quantile regression for high‐dimensional data analysis and variable selection

The present penalized quantile variable selection methods are only applicable to finite number of predictors or do not have oracle property associated with estimator. This technique is considered as an alternative to ordinary least squares regression in case of the outliers and the heavy‐tailed errors existing in linear models. The variable selection through quantile regression with diverging number of parameters is investigated in this paper. The convergence rate of estimator with smoothly clipped absolute deviation penalty function is also studied. Moreover, the oracle property with proper selection of tuning parameter for quantile regression under certain regularity conditions is also established. In addition, the rank correlation screening method is used to accommodate ultra‐high dimensional data settings. Monte Carlo simulations demonstrate finite performance of the proposed estimator. The results of real data reveal that this approach provides substantially more information as compared with ordinary least squares, conventional quantile regression, and quantile lasso.     

Asymptotics and smoothing parameter selection for penalized spline regression with various loss functions

Penalized splines are used in various types of regression analyses, including non‐parametric quantile, robust and the usual mean regression. In this paper, we focus on the penalized spline estimator with general convex loss functions. By specifying the loss function, we can obtain the mean estimator, quantile estimator and robust estimator. We will first study the asymptotic properties of penalized splines. Specifically, we will show the asymptotic bias and variance as well as the asymptotic normality of the estimator. Next, we will discuss smoothing parameter selection for the minimization of the mean integrated squares error. The new smoothing parameter can be expressed uniquely using the asymptotic bias and variance of the penalized spline estimator. To validate the new smoothing parameter selection method, we will provide a simulation. The simulation results show that the consistency of the estimator with the proposed smoothing parameter selection method can be confirmed and that the proposed estimator has better behavior than the estimator with generalized approximate cross‐validation. A real data example is also addressed.     

Estimation of limited dependent variable models by ordinary least squares and the method of moments

Under certain conditions, a broad class of qualitative and limited dependent variable models can be consistently estimated by the method of moments using a non-iterative correction to the ordinary least squares estimator, with only a small loss of efficiency compared to maximum likelihood estimation. The class of models is that obtained from a classical multinormal regression by any type of censoring or truncation and includes the tobit, probit, two-limit probit, truncated regression, and some variants of the sample selection models. The paper derives the estimators and their asymptotic covariance matrices.     

Characterization of the asymptotic distribution of semiparametric M-estimators

This paper develops a concrete formula for the asymptotic distribution of two-step, possibly non-smooth semiparametric M-estimators under general misspecification. Our regularity conditions are relatively straightforward to verify and also weaker than those available in the literature. The first-stage nonparametric estimation may depend on finite dimensional parameters. We characterize: (1) conditions under which the first-stage estimation of nonparametric components do not affect the asymptotic distribution, (2) conditions under which the asymptotic distribution is affected by the derivatives of the first-stage nonparametric estimator with respect to the finite-dimensional parameters, and (3) conditions under which one can allow non-smooth objective functions. Our framework is illustrated by applying it to three examples: (1) profiled estimation of a single index quantile regression model, (2) semiparametric least squares estimation under model misspecification, and (3) a smoothed matching estimator.     

Censored regression quantiles with endogenous regressors

This paper develops a semiparametric method for estimation of the censored regression model when some of the regressors are endogenous (and continuously distributed) and instrumental variables are available for them. A “distributional exclusion” restriction is imposed on the unobservable errors, whose conditional distribution is assumed to depend on the regressors and instruments only through a lower-dimensional “control variable,” here assumed to be the difference between the endogenous regressors and their conditional expectations given the instruments. This assumption, which implies a similar exclusion restriction for the conditional quantiles of the censored dependent variable, is used to motivate a two-stage estimator of the censored regression coefficients. In the first stage, the conditional quantile of the dependent variable given the instruments and the regressors is nonparametrically estimated, as are the first-stage reduced-form residuals to be used as control variables. The second-stage estimator is a weighted least squares regression of pairwise differences in the estimated quantiles on the corresponding differences in regressors, using only pairs of observations for which both estimated quantiles are positive (i.e., in the uncensored region) and the corresponding difference in estimated control variables is small. The paper gives the form of the asymptotic distribution for the proposed estimator, and discusses how it compares to similar estimators for alternative models.     

Robust explicit estimators of Weibull parameters

The Weibull distribution plays a central role in modeling duration data. Its maximum likelihood estimator is very sensitive to outliers. We propose three robust and explicit Weibull parameter estimators: the quantile least squares, the repeated median and the median/Q _n estimator. We derive their breakdown point, influence function, asymptotic variance and study their finite sample properties in a Monte Carlo study. The methods are illustrated on real lifetime data affected by a recording error.     

Inference on endogenously censored regression models using conditional moment inequalities

Under a quantile restriction, randomly censored regression models can be written in terms of conditional moment inequalities. We study the identified features of these moment inequalities with respect to the regression parameters where we allow for covariate dependent censoring, endogenous censoring and endogenous regressors. These inequalities restrict the parameters to a set. We show regular point identification can be achieved under a set of interpretable sufficient conditions. We then provide a simple way to convert conditional moment inequalities into unconditional ones while preserving the informational content. Our method obviates the need for nonparametric estimation, which would require the selection of smoothing parameters and trimming procedures. Without the point identification conditions, our objective function can be used to do inference on the partially identified parameter. Maintaining the point identification conditions, we propose a quantile minimum distance estimator which converges at the parametric rate to the parameter vector of interest, and has an asymptotically normal distribution. A small scale simulation study and an application using drug relapse data demonstrate satisfactory finite sample performance.     

Common correlated effects estimation of heterogeneous dynamic panel quantile regression models

This paper proposes a quantile regression estimator for a heterogeneous panel model with lagged dependent variables and interactive effects. The paper adopts the Common Correlated Effects (CCE) approach proposed in the literature and demonstrates that the extension to the estimation of dynamic quantile regression models is feasible under similar conditions to the ones used in the literature. The new quantile regression estimator is shown to be consistent and its asymptotic distribution is derived. Monte Carlo studies are carried out to study the small sample behavior of the proposed approach. The evidence shows that the estimator can significantly improve on the performance of existing estimators as long as the time series dimension of the panel is large. We present an application to the evaluation of Time-of-Use pricing using a large randomized control trial.     

Local linear M-estimation for spatial processes in fixed-design models

We investigate the local linear M-estimation for regression in a fixed-design model when the errors are from a strongly mixing random field. We establish the weak and strong consistency as well as the asymptotic normality of the local linear M-estimator. The conditions on ρ(·) used in this paper are mild and allow many important special cases such as the least square estimator and the least absolute distance estimator.     

Posterior Inference in Bayesian Quantile Regression with Asymmetric Laplace Likelihood

The paper discusses the asymptotic validity of posterior inference of pseudo‐Bayesian quantile regression methods with complete or censored data when an asymmetric Laplace likelihood is used. The asymmetric Laplace likelihood has a special place in the Bayesian quantile regression framework because the usual quantile regression estimator can be derived as the maximum likelihood estimator under such a model, and this working likelihood enables highly efficient Markov chain Monte Carlo algorithms for posterior sampling. However, it seems to be under‐recognised that the stationary distribution for the resulting posterior does not provide valid posterior inference directly. We demonstrate that a simple adjustment to the covariance matrix of the posterior chain leads to asymptotically valid posterior inference. Our simulation results confirm that the posterior inference, when appropriately adjusted, is an attractive alternative to other asymptotic approximations in quantile regression, especially in the presence of censored data.     

The asymptotic behaviour of the estimated generalized least squares method in the linear regression model

Abstract  In the linear regression model the generalized least squares (GLS) method is only applicable if the covariance matrix of the errors is known but for a scalar factor. Otherwise an estimator for this matrix has to be used. Then we speak of the estimated generalized least squares (EGLS) method. In this paper the asymptotic behaviour of both methods is compared. Results are applied to some standard models commonly used in econometrics     

Nonstationary nonlinear heteroskedasticity in regression

This paper considers the regression with errors having nonstationary nonlinear heteroskedasticity. For both the usual stationary regression and the nonstationary cointegrating regression, we develop the asymptotic theories for the least squares methods in the presence of conditional heterogeneity given as a nonlinear function of an integrated process. In particular, we show that the nonstationarity of volatility in the regression errors may induce spuriousness of the underlying regression, if excessive nonstationary volatility is present in the errors. Mild nonstationary volatilities do not render the underlying regression spurious, but their presence makes the least squares estimator asymptotically biased and inefficient and the usual chi-square test invalid.     

The adequacy of asymptotic approximations in the near-integrated autoregressive model with dependent errors

We consider the normalized least squares estimator of the parameter in a nearly integrated first-order autoregressive model with dependent errors. In a first step we consider its asymptotic distribution as well as asymptotic expansion up to order O_p(T⁻¹). We derive a limiting moment generating function which enables us to calculate various distributional quantities by numerical integration. A simulation study is performed to assess the adequacy of the asymptotic distribution when the errors are correlated. We focus our attention on two leading cases: MA(1) errors and AR(1) errors. The asymptotic approximations are shown to be inadequate as the MA root gets close to −1 and as the AR root approaches either −1 or 1. Our theoretical analysis helps to explain and understand the simulation results of Schwert (1989) and DeJong, Nankervis, Savin, and Whiteman (1992) concerning the size and power of Phillips and Perron's (1988) unit root test. A companion paper, Nabeya and Perron (1994), presents alternative asymptotic frameworks in the cases where the usual asymptotic distribution fails to provide an adequate approximation to the finite-sample distribution.     

Focused information criterion and model averaging in censored quantile regression

In this paper, we study model selection and model averaging for quantile regression with randomly right censored response. We consider a semi-parametric censored quantile regression model without distribution assumptions. Under general conditions, a focused information criterion and a frequentist model averaging estimator are proposed, and theoretical properties of the proposed methods are established. The performances of the procedures are illustrated by extensive simulations and the primary biliary cirrhosis data.     

R-estimation of the parameters of a multiple regression model with measurement errors

We consider the theory of R-estimation of the regression parameters of a multiple regression models with measurement errors. Using the standard linear rank statistics, R-estimators are defined and their asymptotic properties are studied as robust alternatives to the least squares estimator. This paper fills the gap of the rank theory for the estimation of regression parameters with measurement error models. Some simulation results are presented to show the effectiveness of the R-estimators.     

BALANCED VARIABLE ADDITION IN LINEAR MODELS

This paper studies what happens when we move from a short regression to a long regression in a setting where both regressions are subject to misspecification. In this setup, the least‐squares estimator in the long regression may have larger inconsistency than the least‐squares estimator in the short regression. We provide a simple interpretation for the comparison of the inconsistencies and study under which conditions the additional regressors in the long regression represent a “balanced addition” to the short regression.     

Partially generalized least squares and two-stage least squares estimators

A class of partially generalized least squares estimators and a class of partially generalized two-stage least squares estimators in regression models with heteroscedastic errors are proposed. By using these estimators a researcher can attain higher efficiency than that attained by the least squares or the two-stage least squares estimators without explicitly estimating each component of the heteroscedastic variances. However, the efficiency is not as high as that of the generalized least squares or the generalized two-stage least squares estimator calculated using the knowledge of the true variances. Hence the use of the term partial.     

On solving bias‐corrected non‐linear estimation equations with an application to the dynamic linear model

In a seminal paper, Mak, Journal of the Royal Statistical Society B, 55, 1993, 945, derived an efficient algorithm for solving non‐linear unbiased estimation equations. In this paper, we show that when Mak's algorithm is applied to biased estimation equations, it results in the estimates that would come from solving a bias‐corrected estimation equation, making it a consistent estimator if regularity conditions hold. In addition, the properties that Mak established for his algorithm also apply in the case of biased estimation equations but for estimates from the bias‐corrected equations. The marginal likelihood estimator is obtained when the approach is applied to both maximum likelihood and least squares estimation of the covariance matrix parameters in the general linear regression model. The new approach results in two new estimators when applied to the profile and marginal likelihood functions for estimating the lagged dependent variable coefficient in the dynamic linear regression model. Monte Carlo simulation results show the new approach leads to a better estimator when applied to the standard profile likelihood. It is therefore recommended for situations in which standard estimators are known to be biased.     

Functional Coefficient Cointegration Models Subject to Time–Varying Volatility with an Application to the Purchasing Power Parity

This paper analyses functional coefficient cointegration models with both stationary and non‐stationary covariates, allowing time‐varying (unconditional) volatility of a general form. The conventional kernel weighted least squares (KLS) estimator is subject to potential efficiency loss, and can be improved by an adaptive kernel weighted least squares (AKLS) estimator that adapts to heteroscedasticity of unknown form. The AKLS estimator is shown to be as efficient as the oracle generalized kernel weighted least squares estimator asymptotically, and can achieve significant efficiency gain relative to the KLS estimator in finite samples. An illustrative example is provided by investigating the Purchasing Power Parity hypothesis.     

Instrumental variable estimation of nonseparable models

There are many environments where knowledge of a structural relationship is required to answer questions of interest. Also, nonseparability of a structural disturbance is a key feature of many models. Here, we consider nonparametric identification and estimation of a model that is monotonic in a nonseparable scalar disturbance, which disturbance is independent of instruments. This model leads to conditional quantile restrictions. We give local identification conditions for the structural equations from those quantile restrictions. We find that a modified completeness condition is sufficient for local identification. We also consider estimation via a nonparametric minimum distance estimator. The estimator minimizes the sum of squares of predicted values from a nonparametric regression of the quantile residual on the instruments. We show consistency of this estimator.