Asymptotically optimal estimation of smooth functionals for interval censoring, part 1

Estimation in the interval censoring model is considered. A class of smooth functionals is introduced, of which the mean is an example. The asymptotic information lower bound for such functionals can be represented as an inner product of two functions. In case 1, i.e. one observation time per unobservable event time, both functions can be given explicitly. We mainly consider case 2, with two observation times for each unobservable event time, in the situation that the observation times can not become arbitrarily close to each other. For case 2, one of the functions in the inner product can only be given implicitly as solution to a Fredholm integral equation. We study properties of this solution and, in a sequel to this paper, prove that the nonparametric maximum likelihood estimator of the functional asymptotically reaches the information lower bound.     

Asymptotically optimal estimation of smooth functionals for interval censoring, part 2

Estimation in the interval censoring model is considered. A class of smooth functionals is introduced, of which the mean is an example. We consider case 2, with two observation times for each unobservable event time, in the situation that the observation times cannot become arbitrarily close to each other. It is proved that the nonparametric maximum likelihood estimator of the functional asymptotically reaches the information lower bound.     

Some model-restricted shrinkage estimators for contingency tables with missing data

For contingency tables with extensive missing data, the unrestricted MLE under the saturated model, computed by the EM algorithm, is generally unsatisfactory. In this case, it may be better to fit a simpler model by imposing some restrictions on the parameter space. Perlman and Wu (1999) propose lattice conditional independence (LCI) models for contingency tables with arbitrary missing data patterns. When this LCI model fits well, the restricted MLE under the LCI model is more accurate than the unrestricted MLE under the saturated model, but not in general. Here we propose certain empirical Bayes (EB) estimators that adaptively combine the best features of the restricted and unrestricted MLEs. These EB estimators appear to be especially useful when the observed data is sparse, even in cases where the suitability of the LCI model is uncertain. We also study a restricted EM algorithm (called the ER algorithm) with similar desirable features. Received: July 1999     

Smooth estimators for estimating order restricted scale parameters of two gamma distributions

We consider the problem of component-wise estimation of ordered scale parameters of two gamma populations, when it is known apriori which population corresponds to each ordered parameter. Under the scale equivariant squared error loss function, smooth estimators that improve upon the best scale equivariant estimators are derived. These smooth estimators are shown to be generalized Bayes with respect to a non-informative prior. Finally, using Monte Carlo simulations, these improved smooth estimators are compared with the best scale equivariant estimators, their non-smooth improvements obtained in Vijayasree, Misra & Singh (1995), and the restricted maximum likelihood estimators. Acknowledgments. Authors are thankful to a referee for suggestions leading to improved presentation.     

Performance of empirical Bayes estimators of random coefficients in multilevel analysis: Some results for the random intercept-only model

For a multilevel model with two levels and only a random intercept, the quality of different estimators of the random intercept is examined. Analytical results are given for the marginal model interpretation where negative estimates of the variance components are allowed for. Except for four or five level-2 units, the Empirical Bayes Estimator (EBE) has a lower average Bayes risk than the Ordinary Least Squares Estimator (OLSE). The EBEs based on restricted maximum likelihood (REML) estimators of the variance components have a lower Bayes risk than the EBEs based on maximum likelihood (ML) estimators. For the hierarchical model interpretation, where estimates of the variance components are restricted being positive, Monte Carlo simulations were done. In this case the EBE has a lower average Bayes risk than the OLSE, also for four or five level-2 units. For large numbers of level-1 (30) or level-2 units (100), the performances of REML-based and ML-based EBEs are comparable. For small numbers of level-1 (10) and level-2 units (25), the REML-based EBEs have a lower Bayes risk than ML-based EBEs only for high intraclass correlations (0.5).     

Nonparametric estimation for a current status right‐censored data model

We consider the Case 1 interval censoring approach for right‐censored survival data. An important feature of the model is that right‐censored event times are not observed exactly, but at some inspection times. The model covers as particular cases right‐censored data, current status data, and life table survival data with a single inspection time. We discuss the nonparametric estimation approach and consider three nonparametric estimators for the survival function of failure time: maximum likelihood, pseudolikelihood, and the naïve estimator. We establish strong consistency of the estimators with the L₁ rate of convergence. Simulation results confirm consistency of the estimators.     

Smooth estimation of a monotone hazard and a monotone density under random censoring

We consider kernel smoothed Grenander‐type estimators for a monotone hazard rate and a monotone density in the presence of randomly right censored data. We show that they converge at rate n^2/5 and that the limit distribution at a fixed point is Gaussian with explicitly given mean and variance. It is well known that standard kernel smoothing leads to inconsistency problems at the boundary points. It turns out that, also by using a boundary correction, we can only establish uniform consistency on intervals that stay away from the end point of the support (although we can go arbitrarily close to the right boundary).     

Nonparametric estimation of variance, skewness and kurtosis of the distribution of a statistic by jackknife and bootstrap techniques

While jackknife and bootstrap estimates of the variance of a statistic are well–known, the author extends these nonparametric maximum likelihood techniques to the estimation of skewness and kurtosis. In addition to the usual negative jackknife also a positive jackknife as proposed by BERAN (1984) receives interest in this work. The performance of the methods is investigated by a Monte Carlo study for Kendall's tau in various situations likely to occur in practice. Possible applications of these developments are discussed.     

Asymptotic confidence intervals for the length of the shortt under random censoring

A short t of a one dimensional probability distribution is defined to be an interval which has at least probability t and minimal length. The length of a show and its obvious estimator are significant measures of scale of a distribution and the corresponding random sample, respectively. In this note a non-parametric asymptotic confidence interval for the length of the (uniqueness is assumed) short t is established in the random censorship from the right model. The estimator of the length of the short t is based on the product-limit (PL) estimator of the unknown distribution function. The proof of the result mainly follows from an appropriate combination of the Glivenko-Cantelli theorem and the functional central limit theorem for the PL estimator.     

Asymptotic normality of the NPMLE of linear functionals for interval censored data, case 1

We give a new proof of the asymptotic normality of a class of linear functionals of the nonparametric maximum likelihood estimator (NPMLE) of a distribution function with "case 1" interval censored data. In particular our proof simplifies the proof of asymptotic normality of the mean given in Groeneboom and Wellner (1992). The proof relies strongly on a rate of convergence result due to van de Geer (1993), and methods from empirical process theory.     

On a Problem of Robbins

An early example of a compound decision problem of Robbins (1951) is employed to illustrate some features of the development of empirical Bayes methods. Our primary objective is to draw attention to the constructive role that the nonparametric maximum likelihood estimator for mixture models introduced by Kiefer & Wolfowitz (1956) can play in these developments.     

Asymptotically optimal nonparametric one- and two-way analysis of variance tests for the log linear model under censoring

We considerr ×c populations with failure ratesλ _ij(t) satisfying the condition

Evaluating the effectiveness of Washington state repeated job search services on the employment rate of prime-age female welfare recipients

This paper uses an unbalanced panel dataset to evaluate how repeated job search services (JSS) and personal characteristics affect the employment rate of the prime-age female welfare recipients in the State of Washington. We propose a transition probability model to take into account issues of sample attrition, sample refreshment and duration dependence. We also generalize Honoré and Kyriazidou’s [Honoré, B.E., Kyriazidou, E., 2000. Panel data discrete choice models with lagged dependent variables. Econometrica 68 (4), 839–874] conditional maximum likelihood estimator to allow for the presence of individual-specific effects. A limited information test is suggested to test for selection issues in non-experimental data. The specification tests indicate that the (conditional on the set of the confounding variables considered) assumptions of no selection due to unobservables and/or no unobserved individual-specific effects are not violated. Our findings indicate that the first job search service does have positive and significant impacts on the employment rate. However, providing repeated JSS to the same client has no significant impact. Further, we find that there are significant experience-enhancing effects. These findings suggest that providing one job search services training to individuals may have a lasting impact on raising their employment rates.     

Estimating the Derivative of a Convex Density

We obtain a relation between the time between two bird-catchings and the total resting period of a bird, leading to the problem of estimating the derivative of a convex density. We state a fundamental result on the nonparametric maximum likelihood estimator of a convex density. Further, we derive the optimal rate in the minimax risk sense for estimating the derivative of a convex density.     

On the optimum number of levels in the one-way model with random effects

The optimal number of levels is studied for the one-way random model with normally distributed effects. The optimum criteria used are based on the variances of the traditional analysis of variance estimators of the variance components. Exact solutions are compared to earlier results based on lower bounds of the sampling variances. Comparisons are also made to the large-sample variances of the estimates based on restricted maximum likelihood. Received February 2002     

Nonparametric identification and estimation of current status data in the presence of death

We present a nonparametric study of current status data in the presence of death. Such data arise from biomedical investigations in which patients are examined for the onset of a certain disease, for example, tumor progression, but may die before the examination. A key difference between such studies on human subjects and the survival–sacrifice model in animal carcinogenicity experiments is that, due to ethical and perhaps technical reasons, deceased human subjects are not examined, so that the information on their disease status is lost. We show that, for current status data with death, only the overall and disease‐free survival functions can be identified, whereas the cumulative incidence of the disease is not identifiable. We describe a fast and stable algorithm to estimate the disease‐free survival function by maximizing a pseudo‐likelihood with plug‐in estimates for the overall survival rates. It is then proved that the global rate of convergence for the nonparametric maximum pseudo‐likelihood estimator is equal to O_p(n^?1/3) or the convergence rate of the estimated overall survival function, whichever is slower. Simulation studies show that the nonparametric maximum pseudo‐likelihood estimators are fairly accurate in small‐ to medium‐sized samples. Real data from breast cancer studies are analyzed as an illustration.     

Pseudo empirical likelihood method in the presence of missing data

In this paper, we propose an estimator for the population mean when some observations on the study and auxiliary variables are missing from the sample. The proposed estimator is valid for any unequal probability sampling design, and is based upon the pseudo empirical likelihood method. The proposed estimator is compared with other estimators in a simulation study.     

Specification of Distance Functions Using Semi- and Nonparametric Methods with an Application to the Dynamic Performance of Eastern and Western European Air Carriers

In this paper we examine the productive performance of a group of three East European carriers and compare it to thirteen of their West European competitors during the period 1977–1990. We first model the multiple output/multiple input technology with a stochastic distance frontier using recently developed semiparametric efficient methods. The endogeneity of multiple outputs is addressed in part by introducing multivariate kernel estimators for the joint distribution of the multiple outputs and potentially correlated firm random effects. We augment estimates from our semiparametric stochastic distance function with nonparametric distance function methods, using linear programming techniques, as well as with extended decomposition methods, based on the Malmquist index number. Both semi- and nonparametric methods indicate significant slack in resource utilization in the East European carriers relative to their Western counterparts, and limited convergence in efficiency or technical change between them. The implications are rather stark for the long run viability of the East European carriers in our sample.     

The density of the parameter estimators when the observations are distributed exponentially

We present the probability density of parameter estimators whenN independent variables are observed, each of them distributed according to the exponential low (with some parameters to be estimated). The numberN is supposed to be small. Typically, such an experimental situation arises in problems of software reliability, another case is a small sample in the GLIM modeling. The considered estimator is defined by the maximum of the posterior probability density; it is equal to the maximum likelihood estimator when the prior is uniform. The exact density is obtained, and its approximation is discussed in accordance with some information-geometric considerations. The main body of the paper has been prepared during the author’s visit in LMC/IMAG Grenoble, France, on the invitation of Université Joseph Fourier in January 1994.