On kernel nonparametric regression designed for complex survey data

In this article, we consider nonparametric regression analysis between two variables when data are sampled through a complex survey. While nonparametric regression analysis has been widely used with data that may be assumed to be generated from independently and identically distributed (iid) random variables, the methods and asymptotic analyses established for iid data need to be extended in the framework of complex survey designs. Local polynomial regression estimators are studied, which include as particular cases design-based versions of the Nadaraya–Watson estimator and of the local linear regression estimator. In this paper, special emphasis is given to the local linear regression estimator. Our estimators incorporate both the sampling weights and the kernel weights. We derive the asymptotic mean squared error (MSE) of the kernel estimators using a combined inference framework, and as a corollary consistency of the estimators is deduced. Selection of a bandwidth is necessary for the resulting estimators; an optimal bandwidth can be determined, according to the MSE criterion in the combined mode of inference. Simulation experiments are conducted to illustrate the proposed methodology and an application with the Canadian survey of labour and income dynamics is presented.     

Detections of changes in return by a wavelet smoother with conditional heteroscedastic volatility

In this paper, we propose two estimators, an integral estimator and a discretized estimator, for the wavelet coefficient of regression functions in nonparametric regression models with heteroscedastic variance. These estimators can be used to test the jumps of the regression function. The model allows for lagged-dependent variables and other mixing regressors. The asymptotic distributions of the statistics are established, and the asymptotic critical values are analytically obtained from the asymptotic distribution. We also use the test to determine consistent estimators for the locations of change points. The jump sizes and locations of change points can be consistently estimated using wavelet coefficients, and the convergency rates of these estimators are derived. We perform some Monte Carlo simulations to check the powers and sizes of the test statistics. Finally, we give practical examples in finance and economics to detect changes in stock returns and short-term interest rates using the empirical wavelet method.     

An unexpected property of minimax estimation in the relative squared error approach to linear regression analysis

An unexpected property of the relative squared error approach to linear regression analysis is derived: It is shown that an estimator being minimax among all linear affine estimators is also minimax in the set of all estimators. Two illustrative special cases are mentioned, where a generalized least squares estimator and a general ridge or Kuks-Olman estimator turn out to be minimax.     

Effective nonparametric estimation in the case of severely discretized data

Often economic data are discretized or rounded to some extent. This paper proposes a regression and a density estimator that work especially well when discretization causes conventional kernel-based estimators to behave poorly. The estimator proposed here is a weighted average of neighboring frequency estimators, and the weights are composed of cubic B-splines. Interestingly, we show that this estimator can have both a smaller bias and variance than frequency estimators. As a means to obtain asymptotic normality and rates of convergence, we assume that the discreteness becomes finer as the sample size increases.     

Semiparametric robust estimation of truncated and censored regression models

Many estimation methods of truncated and censored regression models such as the maximum likelihood and symmetrically censored least squares (SCLS) are sensitive to outliers and data contamination as we document. Therefore, we propose a semiparametric general trimmed estimator (GTE) of truncated and censored regression, which is highly robust but relatively imprecise. To improve its performance, we also propose data-adaptive and one-step trimmed estimators. We derive the robust and asymptotic properties of all proposed estimators and show that the one-step estimators (e.g., one-step SCLS) are as robust as GTE and are asymptotically equivalent to the original estimator (e.g., SCLS). The finite-sample properties of existing and proposed estimators are studied by means of Monte Carlo simulations.     

Distribution free estimation of heteroskedastic binary response models using Probit/Logit criterion functions

In this paper estimators for distribution free heteroskedastic binary response models are proposed. The estimation procedures are based on relationships between distribution free models with a conditional median restriction and parametric models (such as Probit/Logit) exhibiting (multiplicative) heteroskedasticity. The first proposed estimator is based on the observational equivalence between the two models, and is a semiparametric sieve estimator (see, e.g. Gallant and Nychka (1987), Ai and Chen (2003) and Chen et al. (2005)) for the regression coefficients, based on maximizing standard Logit/Probit criterion functions, such as NLLS and MLE. This procedure has the advantage that choice probabilities and regression coefficients are estimated simultaneously. The second proposed procedure is based on the equivalence between existing semiparametric estimators for the conditional median model (,  and ) and the standard parametric (Probit/Logit) NLLS estimator. This estimator has the advantage of being implementable with standard software packages such as Stata. Distribution theory is developed for both estimators and a Monte Carlo study indicates they both perform well in finite samples.     

Asymptotic properties of the efficient estimators for cointegrating regression models with serially dependent errors

In this paper, we analytically investigate three efficient estimators for cointegrating regression models: Phillips and Hansen’s [Phillips, P.C.B., Hansen, B.E., 1990. Statistical inference in instrumental variables regression with I(1) processes. Review of Economic Studies 57, 99–125] fully modified OLS estimator, Park’s [Park, J.Y., 1992. Canonical cointegrating regressions. Econometrica 60, 119–143] canonical cointegrating regression estimator, and Saikkonen’s [Saikkonen, P., 1991. Asymptotically efficient estimation of cointegration regressions. Econometric Theory 7, 1–21] dynamic OLS estimator. We consider the case where the regression errors are moderately serially correlated and the AR coefficient in the regression errors approaches 1 at a rate slower than 1/T$1/T$, where T$T$ represents the sample size. We derive the limiting distributions of the efficient estimators under this system and find that they depend on the approaching rate of the AR coefficient. If the rate is slow enough, efficiency is established for the three estimators; however, if the approaching rate is relatively faster, the estimators will have the same limiting distribution as the OLS estimator. For the intermediate case, the second-order bias of the OLS estimator is partially eliminated by the efficient methods. This result explains why, in finite samples, the effect of the efficient methods diminishes as the serial correlation in the regression errors becomes stronger. We also propose to modify the existing efficient estimators in order to eliminate the second-order bias, which possibly remains in the efficient estimators. Using Monte Carlo simulations, we demonstrate that our modification is effective when the regression errors are moderately serially correlated and the simultaneous correlation is relatively strong.     

Estimators of Regression Parameters for Truncated and Censored Data

Estimators of parameters in semi-parametric left truncated and right censored regression models are proposed. In contrast to the majority of existing estimators, the proposed estimators do not require the error term of the regression model to have a symmetric distribution. In addition the estimators use asymmetric “trimming” of observations. Consistency and asymptotic normality of the estimators are shown. Finite sample properties are considered in a small simulation study. For the left truncated case, an empirical application illustrates the usefulness of the estimator.     

Mixing of direct, ratio, and product method estimators

In a paper by S rivenkataramana T racy [4], four methods of estimating a population total Y with the use of an auxiliary variable were introduced, given a random sample without replacement from that population. These methods were "built around the idea that estimating the population total is essentially equivalent to estimating the total corresponding to the non-sample units, since that corresponding to the sample units is known once the sample is drawn and measurements are made on it."
However, in the case of small sampling fractions the nonsample units constitute most of the population and no great improvement over the traditional estimators is to be expected. Therefore the methods are compared with the existing estimators and it turns out that they are special cases of the "mixing estimators", introduced in this paper. The latter estimators can be made asymptotically equivalent to the regression estimator and are therefore asymptotically superior to all other estimators. An exact comparison is carried out on the artificial example given in [4]. The statement in this paper that "the proposed estimators are to be preferred to the regression estimator for., superiority of performance in the case of small samples" is evidently misleading. Finally a comparison is made with other "mixing-type" estimators, that can prove very useful in practice.     

Sieve estimation of panel data models with cross section dependence

In this paper we consider the problem of estimating semiparametric panel data models with cross section dependence, where the individual-specific regressors enter the model nonparametrically whereas the common factors enter the model linearly. We consider both heterogeneous and homogeneous regression relationships when both the time and cross-section dimensions are large. We propose sieve estimators for the nonparametric regression functions by extending Pesaran’s (2006) common correlated effect (CCE) estimator to our semiparametric framework. Asymptotic normal distributions for the proposed estimators are derived and asymptotic variance estimators are provided. Monte Carlo simulations indicate that our estimators perform well in finite samples.     

Calibration Estimation in Survey Sampling

Calibration estimation, where the sampling weights are adjusted to make certain estimators match known population totals, is commonly used in survey sampling. The generalized regression estimator is an example of a calibration estimator. Given the functional form of the calibration adjustment term, we establish the asymptotic equivalence between the functional-form calibration estimator and an instrumental variable calibration estimator where the instrumental variable is directly determined from the functional form in the calibration equation. Variance estimation based on linearization is discussed and applied to some recently proposed calibration estimators. The results are extended to the estimator that is a solution to the calibrated estimating equation. Results from a limited simulation study are presented.     

Semiparametric econometric estimators for a truncated regression model: a review with an extension

Econometric estimators for a truncated regression model are reviewed. For each estimator, the motivations, the key assumptions, the asymptotic distribution and estimates for the asymptotic variance matrix are presented; also a new estimator is suggested. We select five practical estimators among those, and compare them through a Monte Carlo study where the response variable is simulated but the covariates are drawn from a real data set. Some practical and computational issues are addressed as well.     

Estimating Mann–Whitney‐type Causal Effects

Mann–Whitney‐type causal effects are generally applicable to outcome variables with a natural ordering, have been recommended for clinical trials because of their clinical relevance and interpretability and are particularly useful in analysing an ordinal composite outcome that combines an original primary outcome with death and possibly treatment discontinuation. In this article, we consider robust and efficient estimation of such causal effects in observational studies and clinical trials. For observational studies, we propose and compare several estimators: regression estimators based on an outcome regression (OR) model or a generalised probabilistic index (GPI) model, an inverse probability weighted estimator based on a propensity score model and two doubly robust (DR), locally efficient estimators. One of the DR estimators involves a propensity score model and an OR model, is consistent and asymptotically normal under the union of the two models and attains the semiparametric information bound when both models are correct. The other DR estimator has the same properties with the OR model replaced by a GPI model. For clinical trials, we extend an existing augmented estimator based on a GPI model and propose a new one based on an OR model. The methods are evaluated and compared in simulation experiments and applied to a clinical trial in cardiology and an observational study in obstetrics.     

Model reference adaptive systems applied to regression analyses

The adaptive estimation procedure of model reference adaptive systems is modified and applied to linear models. In general the principle can be used for almost any time series model. Because of the recursive nature of the resulting estimator, it is computationally appealing, especially when a time series is considered as a flow of data. In addition, the estimator turns out to have certain statistical optimality properties.
In the linear regression setting, Ridge estimators turn out to constitute a subclass of the adaptive estimators considered, whereas for unknown measurement variance, the resulting estimators are related to J ames -S tkin type estimators, and have better properties than the latter. The estimator is shown to be strongly consistent and to converge in law to a normal variate under the standard assumptions of linear models. Further it is shown to be admissible and minimax in restricted parameter spaces. The connection between K alman filters and the classical least-squares estimator is also pointed out.     

Semiparametric estimation of logistic regression model with missing covariates and outcome

We consider a semiparametric method to estimate logistic regression models with missing both covariates and an outcome variable, and propose two new estimators. The first, which is based solely on the validation set, is an extension of the validation likelihood estimator of Breslow and Cain (Biometrika 75:11–20, 1988). The second is a joint conditional likelihood estimator based on the validation and non-validation data sets. Both estimators are semiparametric as they do not require any model assumptions regarding the missing data mechanism nor the specification of the conditional distribution of the missing covariates given the observed covariates. The asymptotic distribution theory is developed under the assumption that all covariate variables are categorical. The finite-sample properties of the proposed estimators are investigated through simulation studies showing that the joint conditional likelihood estimator is the most efficient. A cable TV survey data set from Taiwan is used to illustrate the practical use of the proposed methodology.     

Small sample corrections for LTS and MCD

The least trimmed squares estimator and the minimum covariance determinant estimator [6] are frequently used robust estimators of regression and of location and scatter. Consistency factors can be computed for both methods to make the estimators consistent at the normal model. However, for small data sets these factors do not make the estimator unbiased. Based on simulation studies we therefore construct formulas which allow us to compute small sample correction factors for all sample sizes and dimensions without having to carry out any new simulations. We give some examples to illustrate the effect of the correction factor.     

Properties of shrinkage estimators in linear regression when disturbances are not normal

This paper considers a class of recently developed biased estimators of regression coefficients and studies its sampling properties when the disturbances are not normally distributed. It has been found that the conditions of dominance of these estimators over the least squares estimator, under non-normality, are quite different than their well-known dominance conditions under normality. Some implications of the results are also discussed.     

Partial rank estimation of duration models with general forms of censoring

In this paper we propose estimators for the regression coefficients in censored duration models which are distribution free, impose no parametric specification on the baseline hazard function, and can accommodate general forms of censoring. The estimators are shown to have desirable asymptotic properties and Monte Carlo simulations demonstrate good finite sample performance. Among the data features the new estimators can accommodate are covariate-dependent censoring, double censoring, and fixed (individual or group specific) effects. We also examine the behavior of the estimator in an empirical illustration.     

Maximum likelihood and restricted maximum likelihood estimation for a class of Gaussian Markov random fields

This work describes a Gaussian Markov random field model that includes several previously proposed models, and studies properties of its maximum likelihood (ML) and restricted maximum likelihood (REML) estimators in a special case. Specifically, for models where a particular relation holds between the regression and precision matrices of the model, we provide sufficient conditions for existence and uniqueness of ML and REML estimators of the covariance parameters, and provide a straightforward way to compute them. It is found that the ML estimator always exists while the REML estimator may not exist with positive probability. A numerical comparison suggests that for this model ML estimators of covariance parameters have, overall, better frequentist properties than REML estimators.     

Mean squared error of ridge estimators in logistic regression

It is well known that the maximum likelihood estimator (MLE) is inadmissible when estimating the multidimensional Gaussian location parameter. We show that the verdict is much more subtle for the binary location parameter. We consider this problem in a regression framework by considering a ridge logistic regression (RR) with three alternative ways of shrinking the estimates of the event probabilities. While it is shown that all three variants reduce the mean squared error (MSE) of the MLE, there is at the same time, for every amount of shrinkage, a true value of the location parameter for which we are overshrinking, thus implying the minimaxity of the MLE in this family of estimators. Little shrinkage also always reduces the MSE of individual predictions for all three RR estimators; however, only the naive estimator that shrinks toward 1/2 retains this property for any generalized MSE (GMSE). In contrast, for the two RR estimators that shrink toward the common mean probability, there is always a GMSE for which even a minute amount of shrinkage increases the error. These theoretical results are illustrated on a numerical example. The estimators are also applied to a real data set, and practical implications of our results are discussed.