Efficiency. of infinite dimensional M- estimators

It is well-known that maximum likelihood estimators are asymptotically normal with covariance equal to the inverse Fisher information in smooth, finite dimensional parametric models. Thus they are asymptotically efficient. A similar phenomenon has been observed for certain infinite dimensional parameter spaces. We give a simple proof of efficiency, starting from a theorem on asymptotic normality of infinite dimensional M -estimators. The proof avoids the explicit calculation of the Fisher information. We also address Hadamard differentiability of the corresponding M -functionals.     

Methods for the Analysis of Explanatory Linear Regression Models with Missing Data Not at Random

Since the work of Little and Rubin (1987) not substantial advances in the analysisof explanatory regression models for incomplete data with missing not at randomhave been achieved, mainly due to the difficulty of verifying the randomness ofthe unknown data. In practice, the analysis of nonrandom missing data is donewith techniques designed for datasets with random or completely random missingdata, as complete case analysis, mean imputation, regression imputation, maximumlikelihood or multiple imputation. However, the data conditions required to minimizethe bias derived from an incorrect analysis have not been fully determined. In thepresent work, several Monte Carlo simulations have been carried out to establishthe best strategy of analysis for random missing data applicable in datasets withnonrandom missing data. The factors involved in simulations are sample size,percentage of missing data, predictive power of the imputation model and existenceof interaction between predictors. The results show that the smallest bias is obtainedwith maximum likelihood and multiple imputation techniques, although with lowpercentages of missing data, absence of interaction and high predictive power ofthe imputation model (frequent data structures in research on child and adolescentpsychopathology) acceptable results are obtained with the simplest regression imputation.     

Some model-restricted shrinkage estimators for contingency tables with missing data

For contingency tables with extensive missing data, the unrestricted MLE under the saturated model, computed by the EM algorithm, is generally unsatisfactory. In this case, it may be better to fit a simpler model by imposing some restrictions on the parameter space. Perlman and Wu (1999) propose lattice conditional independence (LCI) models for contingency tables with arbitrary missing data patterns. When this LCI model fits well, the restricted MLE under the LCI model is more accurate than the unrestricted MLE under the saturated model, but not in general. Here we propose certain empirical Bayes (EB) estimators that adaptively combine the best features of the restricted and unrestricted MLEs. These EB estimators appear to be especially useful when the observed data is sparse, even in cases where the suitability of the LCI model is uncertain. We also study a restricted EM algorithm (called the ER algorithm) with similar desirable features. Received: July 1999     

A local maximum likelihood estimator for Poisson regression

A local maximum likelihood estimator based on Poisson regression is presented as well as its bias, variance and asymptotic distribution. This semiparametric estimator is intended to be an alternative to the Poisson, negative binomial and zero-inflated Poisson regression models that does not depend on regularity conditions and model specification accuracy. Some simulation results are presented. The use of the local maximum likelihood procedure is illustrated on one example from the literature. This procedure is found to perform well. This research was partially supported by Calouste Gulbenkian Foundation and PRODEP III.     

Longitudinal LISREL model estimation from incomplete panel data using the EM algorithm and the Kalman smoother

Longitudinal data sets with the structure T (time points) × N (subjects) are often incomplete because of data missing for certain subjects at certain time points. The EM algorithm is applied in conjunction with the Kalman smoother for computing maximum likelihood estimates of longitudinal LISREL models from varying missing data patterns. The iterative procedure uses the LISREL program in the M-step and the Kalman smoother in the E-step. The application of the method is illustrated by simulating missing data on a data set from educational research.     

A note on multinomial maximum likelihood estimation under ordered restrictions and the EM algorithm

A frequently occurring problem is to find the maximum likelihood estimation (MLE) of p subject to p∈C (C⊂ P the probability vectors in R ^k ). The problem has been discussed by many authors and they mainly focused when p is restricted by linear constraints or log-linear constraints. In this paper, we construct the relationship between the the maximum likelihood estimation of p restricted by p∈C and EM algorithm and demonstrate that the maximum likelihood estimator can be computed through the EM algorithm (Dempster et al. in J R Stat Soc Ser B 39:1–38, 1997). Several examples are analyzed by the proposed method.     

On the analysis of multivariate growth curves

Growth curve data arise when repeated measurements are observed on a number of individuals with an ordered dimension for occasions. Such data appear frequently in almost all fields in which statistical models are used, for instance in medicine, agriculture and engineering. In medicine, for example, more than one variable is often measured on each occasion. However, analyses are usually based on exploration of repeated measurements of only one variable. The consequence is that the information contained in the between-variables correlation structure will be discarded.  In this study we propose a multivariate model based on the random coefficient regression model for the analysis of growth curve data. Closed-form expressions for the model parameters are derived under the maximum likelihood (ML) and the restricted maximum likelihood (REML) framework. It is shown that in certain situations estimated variances of growth curve parameters are greater for REML. Also a method is proposed for testing general linear hypotheses. One numerical example is provided to illustrate the methods discussed. Received: 22 February 1999     

Numerical Maximisation of Likelihood: A Neglected Alternative to EM?

There is by now a long tradition of using the EM algorithm to find maximum‐likelihood estimates (MLEs) when the data are incomplete in any of a wide range of ways, even when the observed‐data likelihood can easily be evaluated and numerical maximisation of that likelihood is available as a conceptually simple route to the MLEs. It is rare in the literature to see numerical maximisation employed if EM is possible. But with excellent general‐purpose numerical optimisers now available free, there is no longer any reason, as a matter of course, to avoid direct numerical maximisation of likelihood. In this tutorial, I present seven examples of models in which numerical maximisation of likelihood appears to have some advantages over the use of EM as a route to MLEs. The mathematical and coding effort is minimal, as there is no need to derive and code the E and M steps, only a likelihood evaluator. In all the examples, the unconstrained optimiser nlm available in R is used, and transformations are used to impose constraints on parameters. I suggest therefore that the following question be asked of proposed new applications of EM: Can the MLEs be found more simply and directly by using a general‐purpose numerical optimiser?     

Asymptotically efficient recursive estimation for incomplete data models using the observed information

For a recursive maximum-likelihood estimator with step lengths decaying as 1/n, an adaptive matrix needs to be incorporated to obtain asymptotic efficiency. Ideally, this matrix should be chosen as the inverse Fisher information matrix, which is usually very difficult to compute for incomplete data models. In this paper we give conditions under which the observed information can be incorporated into the recursive procedure to yield an efficient estimator, and we also investigate the finite sample properties of these estimators by simulation.     

Maximum Likelihood Estimation of a General Unbalanced Spatial Random Effects Model: a Monte Carlo Study

Abstract

This paper discusses the maximum likelihood estimator of a general unbalanced spatial random effects model with normal disturbances, assuming that some observations are missing at random. Monte Carlo simulations show that the maximum likelihood estimator for unbalanced panels performs well and that missing observations affect mainly the root mean square error. As expected, these estimates are less efficient than those based on the unobserved balanced model, especially if the share of missing observations is large or spatial autocorrelation in the error terms is pronounced.

Estimation de vraisemblance maximale d'un modèle général d'effets aléatoires spatiaux déséquilibré: une étude Monte Carlo

RÉSUMÉ La présente communication se penche sur l'estimateur du maximum de vraisemblance d'un modèle général d'effets aléatoires spatiaux déséquilibré avec des perturbations normales, en supposant l'absence aléatoire de certaines observations. Des simulations de Monte Carlo montrent que des groupes déséquilibrés se comporte bien, et que les observations manquantes affectent principalement l'erreur de la moyenne quadratique. Comme prévu, ces évaluations sont moins efficaces que celles qui sont basées sur le modèle équilibré non observé, notamment si la part des observations manquantes est importantes, ou l'on déclare une autocorrélation spatiale dans les termes d'erreur.

Estimación de la probabilidad máxima de un modelo espacial general desequilibrado de efectos al azar: un estudio de Monte Carlo

RÉSUMÉN Este trabajo discute el estimador de probabilidad máxima de un modelo espacial general desequilibrado de efectos al azar con alteraciones normales, suponiendo que faltan algunas observaciones al azar. Las simulaciones de Monte Carlo muestran que el estimador de probabilidad máxima para los paneles desequilibrados funciona satisfactoriamente, y que las observaciones omisas afectan principalmente al error de la media cuadrática. Como se suponía, estas estimaciones son menos eficientes que las basadas en el modelo equilibrado inadvertido, especialmente si la cantidad de omisiones es grande/o la autocorrelación en los términos de error es pronunciada.

     

The efficiency of Spanish arable crop organic farms,a local maximum likelihood approach

The methodologies that have been used in existing research to assess the efficiency with which organic farms are operating are generally based either on the stochastic frontier methodology or on a deterministic non-parametric approach. Recently, Kumbhakar et al. (J Econom 137:1–27, 2007) proposed a new nonparametric, stochastic method based on the local maximum likelihood principle. We use this methodology to compare the efficiency ratings of organic and conventional arable crop farms in the Spanish region of Andalucía. Nonparametrically encompassing the stochastic frontier model is especially useful when comparing the performance of two groups that are likely to be characterized by different production technologies.

Estimation of variances in orthogonal finite discrete spectrum linear regression models

The Invariant Quadratic Estimators, the Maximum Likelihood Estimator (MLE) and Restricted Maximum Likelihood Estimator (REML) of variances in an orthogonal Finite Discrete Spectrum Linear Regression Model (FDSLRM) are derived and the problems of unbiasedness and consistency of these estimators are investigated.Acknowledgement. The research was supported by the grants 1/0272/03, 1/0264/03 and 2/4026/04 of the Slovak Scientific Grant Agency VEGA.