Consistent likelihood-based estimation of a star-shaped distribution

We consider the classical problem of nonparametrically estimating a star-shaped distribution, i.e., a distribution function F on [0,∞) with the property that F(u)/u is nondecreasing on the set {u : F(u) < 1}. This problem is intriguing because of the fact that a well defined maximum likelihood estimator (MLE) exists, but this MLE is inconsistent. In this paper, we argue that the likelihood that is commonly used in this context is somewhat unnatural and propose another, so called ‘smoothed likelihood’. However, also the resulting MLE turns out to be inconsistent. We show that more serious smoothing of the likelihood yields consistent estimators in this model.     

Properties of h-Likelihood Estimators in Clustered Data

We study properties of the maximum h-likelihood estimators for random effects in clustered data. To define optimality in random effects predictions, several foundational concepts of statistics such as likelihood, unbiasedness, consistency, confidence distribution and the Cramer–Rao lower bound are extended. Exact probability statements about interval estimators for random effects can be made asymptotically without a prior assumption. Using the binary-matched pair example, we illustrated that the use of random effects recover information, leading to the boon on estimating treatment effects.     

Optimal estimation under nonstandard conditions

We analyze optimality properties of maximum likelihood (ML) and other estimators when the problem does not necessarily fall within the locally asymptotically normal (LAN) class, therefore covering cases that are excluded from conventional LAN theory such as unit root nonstationary time series. The classical Hájek–Le Cam optimality theory is adapted to cover this situation. We show that the expectation of certain monotone “bowl-shaped” functions of the squared estimation error are minimized by the ML estimator in locally asymptotically quadratic situations, which often occur in nonstationary time series analysis when the LAN property fails. Moreover, we demonstrate a direct connection between the (Bayesian property of) asymptotic normality of the posterior and the classical optimality properties of ML estimators.     

Density estimation in the uniform deconvolution model

We consider the problem of estimating a probability density function based on data that are corrupted by noise from a uniform distribution. The (nonparametric) maximum likelihood estimator for the corresponding distribution function is well defined. For the density function this is not the case. We study two nonparametric estimators for this density. The first is a type of kernel density estimate based on the empirical distribution function of the observable data. The second is a kernel density estimate based on the MLE of the distribution function of the unobservable (uncorrupted) data.     

Median unbiased estimates for M.L.R.-families

Summary Lehmann [p. 83] has shown that some families of probability measures with monotone likelihood ratios (m.l.r.) admit median unbiased estimates which are optimum in the sense that among all median unbiased estimates they minimize the expected loss for any loss function which assumes its minimal value zero for the “true” parameter value and is nondecreasing as the parameter moves away from the true value in either direction. This very strong optimum property was proved under the assumption that all probability measures of the m.l.r.-family have continuous distribution functions, that they are mutually absolutely continuous and that each element of the support is the median of somep-measure of the family. This result does therefore not cover important cases such as the binomial families or thePoisson family. The purpose of the present paper is to show the existence ofrandomized median unbiased estimates with the same optimum property for m.l.r.-families which are closed and connected with respect to the strong topology. Such families are always dominated. We do, however, neither assume that thep-measures are mutually absolutely continuous nor that the distribution functions are continuous. We remark that the use of randomized estimates is indispensable here because nonrandomized median unbiased estimates do not always exist in the general case.     

Estimation of location of discontinuity in a density

Here we propose a few estimators of θ, in addition to those studied in Goria (1978), the point of discontinuity of the probability density $$f(x,\theta ) = \frac{1}{{2\Gamma (\alpha )}}e^{ - |x - \theta |} |x - \theta |^{\alpha - 1} ,$$ for $$0< \alpha< 1, - \infty< x< \infty , - \infty< \theta< \infty .$$ We establish the consistency and the optimality of the Bayes and the maximum probability estimators. Despite their nice properties, these estimators are not easy to compute in this case and their effective computation depends on the knowledge of the exponent α. Hence, we propose another class of estimators, dependent upon the spacings of the observations, computable without actual knowledge of the value of α as long as it is known that α < α₀ < 1: we show that these estimators converge at the best possible rate. We further demonstrate, using a modified version of the maximum probability estimator's technique, that the tails of the density do not substantially effect their efficiency. Finally a bivariate family of densities, having a ridge dependent on the parameter θ, is considered and it is shown that this family exhibits features similar to the univariate case, and thus, the necessary modifications of the arguments of the univariate case are utilized for the estimation of θ in this bivariate example.     

Combining kernel estimators in the uniform deconvolution problem

We construct a density estimator and an estimator of the distribution function in the uniform deconvolution model. The estimators are based on inversion formulas and kernel estimators of the density of the observations and its derivative. Initially the inversions yield two different estimators of the density and two estimators of the distribution function. We construct asymptotically optimal convex combinations of these two estimators. We also derive pointwise asymptotic normality of the resulting estimators, the pointwise asymptotic biases and an expansion of the mean integrated squared error of the density estimator. It turns out that the pointwise limit distribution of the density estimator is the same as the pointwise limit distribution of the density estimator introduced by Groeneboom and Jongbloed (Neerlandica, 57, 2003, 136), a kernel smoothed nonparametric maximum likelihood estimator of the distribution function.     

On the efficiency of least squares estimators in non-linear models

Summary The identity of least squares estimators å and maximum likelihood estimators â is studied in non-linear models of the type z=g(a), where z are observable quantities with a probability density function pr(z). This identity was proved for independent random variables z and for distributions pr(z), of which the arithmetic sample mean is an optimal estimate.     

A bayesian analysis of a random coefficient model in a simple keynesian system

The marginal propensity to consume in a simple Keynesian model is treated as a random coefficient. This gives rise to the problem of quotient of random variables, i.e., the Fieller-Creasy problem. The Bayesian and maximum likelihood estimators are compared in sampling experiments. The Bayesian estimators have smaller mean squared errors than the maximum likelihood estimators. Marginal posterior probability density functions for a given sample are also presented.     

Estimation of order-restricted means of two normal populations under the LINEX loss function

In this paper, the estimation of order-restricted means of two normal distributions is studied under the LINEX loss function, when the variances are unknown and possibly unequal. Under certain sufficient conditions to be described in this paper, the proposed plug-in estimators uniformly perform better than the existing unrestricted maximum likelihood estimators. Further, the restricted maximum likelihood estimators are compared with the unrestricted maximum likelihood estimators under the Pitman nearness criterion. A simulation study is conducted and it is shown that our proposed plug-in estimators perform better than the unrestricted maximum likelihood estimators. An illustrative example of real data analysis is also given to compare the estimators.     

Efficient semiparametric estimation for endogenously stratified regression via smoothed likelihood

This paper presents efficient semiparametric estimators for endogenously stratified regression with two strata, in the case where the error distribution is unknown and the regressors are independent of the error term. The method is based on the use of a kernel-smoothed likelihood function which provides an explicit solution for the maximization problem for the unknown density function without losing information in the asymptotic limit. We consider both standard stratified sampling and variable probability sampling, and allow for the population shares of the strata to be either unknown or known a priori.     

Properties and estimation of asymmetric exponential power distribution

The new distribution class, Asymmetric Exponential Power Distribution (AEPD), proposed in this paper generalizes the class of Skewed Exponential Power Distributions (SEPD) in a way that in addition to skewness introduces different decay rates of density in the left and right tails. Our parametrization provides an interpretable role for each parameter. We derive moments and moment-based measures: skewness, kurtosis, expected shortfall. It is demonstrated that a maximum entropy property holds for the AEPD distributions. We establish consistency, asymptotic normality and efficiency of the maximum likelihood estimators over a large part of the parameter space by dealing with the problems created by non-smooth likelihood function and derive explicit analytical expressions of the asymptotic covariance matrix; where the results apply to the SEPD class they enlarge on the current literature. Also we give a convenient stochastic representation of the distribution; our Monte Carlo study illustrates the theoretical results. We also provide some empirical evidence for the usefulness of employing AEPD errors in GARCH type models for predicting downside market risk of financial assets.     

The density of the parameter estimators when the observations are distributed exponentially

We present the probability density of parameter estimators whenN independent variables are observed, each of them distributed according to the exponential low (with some parameters to be estimated). The numberN is supposed to be small. Typically, such an experimental situation arises in problems of software reliability, another case is a small sample in the GLIM modeling. The considered estimator is defined by the maximum of the posterior probability density; it is equal to the maximum likelihood estimator when the prior is uniform. The exact density is obtained, and its approximation is discussed in accordance with some information-geometric considerations. The main body of the paper has been prepared during the author’s visit in LMC/IMAG Grenoble, France, on the invitation of Université Joseph Fourier in January 1994.     

Estimation of a dynamic stochastic frontier model using likelihood-based approaches

This paper considers a panel stochastic production frontier model that allows the dynamic adjustment of technical inefficiency. In particular, we assume that inefficiency follows an AR(1) process. That is, the current year's inefficiency for a firm depends on its past inefficiency plus a transient inefficiency incurred in the current year. Interfirm variations in the transient inefficiency are explained by some firm-specific covariates. We consider four likelihood-based approaches to estimate the model: the full maximum likelihood, pairwise composite likelihood, marginal composite likelihood, and quasi-maximum likelihood approaches. Moreover, we provide Monte Carlo simulation results to examine and compare the finite-sample performances of the four above-mentioned likelihood-based estimators of the parameters. Finally, we provide an empirical application of a panel of 73 Finnish electricity distribution companies observed during 2008–2014 to illustrate the working of our proposed models.     

On the efficiency of least squares estimators in non-linear models

Summary  The identity of least squares estimators å and maximum likelihood estimators â is studied in non-linear models of the type z = g ( a ), where z are observable quantities with a probability density function pr ( z ). This identity was proved for independent random variables z and for distributions pr ( z ), of which the arithmetic sample mean is an optimal estimate.     

Maximum likelihood and restricted maximum likelihood estimation for a class of Gaussian Markov random fields

This work describes a Gaussian Markov random field model that includes several previously proposed models, and studies properties of its maximum likelihood (ML) and restricted maximum likelihood (REML) estimators in a special case. Specifically, for models where a particular relation holds between the regression and precision matrices of the model, we provide sufficient conditions for existence and uniqueness of ML and REML estimators of the covariance parameters, and provide a straightforward way to compute them. It is found that the ML estimator always exists while the REML estimator may not exist with positive probability. A numerical comparison suggests that for this model ML estimators of covariance parameters have, overall, better frequentist properties than REML estimators.     

Minimax estimation of a cumulative distribution function by converting to a parametric problem

Let X = (X ₁,...,X _n ) be a sample from an unknown cumulative distribution function F defined on the real line . The problem of estimating the cumulative distribution function F is considered using a decision theoretic approach. No assumptions are imposed on the unknown function F. A general method of finding a minimax estimator d(t;X) of F under the loss function of a general form is presented. The method of solution is based on converting the nonparametric problem of searching for minimax estimators of a distribution function to the parametric problem of searching for minimax estimators of the probability of success for a binomial distribution. The solution uses also the completeness property of the class of monotone decision procedures in a monotone decision problem. Some special cases of the underlying problem are considered in the situation when the loss function in the nonparametric problem is defined by a weighted squared, LINEX or a weighted absolute error.     

Sparse estimators and the oracle property,or the return of Hodges’ estimator

We point out some pitfalls related to the concept of an oracle property as used in Fan and Li [2001. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360; 2002. Variable selection for Cox's proportional hazards model and frailty model. Annals of Statistics 30, 74–99; 2004. New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. Journal of the American Statistical Association 99, 710–723] which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li [2001. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360]. We find that this estimator can perform rather poorly in finite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.     

On finite sample properties of alternative estimators of coefficients in a structural equation with many instruments

We compare four different estimation methods for the coefficients of a linear structural equation with instrumental variables. As the classical methods we consider the limited information maximum likelihood (LIML) estimator and the two-stage least squares (TSLS) estimator, and as the semi-parametric estimation methods we consider the maximum empirical likelihood (MEL) estimator and the generalized method of moments (GMM) (or the estimating equation) estimator. Tables and figures of the distribution functions of four estimators are given for enough values of the parameters to cover most linear models of interest and we include some heteroscedastic cases and nonlinear cases. We have found that the LIML estimator has good performance in terms of the bounded loss functions and probabilities when the number of instruments is large, that is, the micro-econometric models with “many instruments” in the terminology of recent econometric literature.     

Nonparametric maximum likelihood estimation in a non locally compact setting

Maximum likelihood estimation is considered in the context of infinite dimensional parameter spaces. It is shown that in some locally convex parameter spaces sequential compactness of the bounded sets ensures the existence of minimizers of objective functions and the consistency of maximum likelihood estimators in an appropriate topology. The theory is applied to revisit some classical problems of nonparametric maximum likelihood estimation, to study location parameters in Banach spaces, and finally to obtain Varadarajan’s theorem on the convergence of empirical measures in the form of a consistency result for a sequence of maximum likelihood estimators. Several parameter spaces sharing the crucial compactness property are identified. This research was supported by grants from the National Sciences and Engineering Research Council of Canada and the Fonds FCAR de la Province de Québec.