Analysis of a mixed model for ordinal data by iterative re-weighted REML

An estimation procedure will be presented for a class of threshold models for ordinal data. These models may include both fixed and random effects with associated components of variance on an underlying scale. The residual error distribution on the underlying scale may be rendered greater flexibility by introducing additional shape parameters, e.g. a kurtosis parameter or parameters to model heterogeneous residual variances as a function of factors and covariates. The estimation procedure is an extension of an iterative re-weighted restricted maximum likelihood procedure, originally developed for generalized linear mixed models. This procedure will be illustrated with a practical problem involving damage to potato tubers and with data from animal breeding and medical research from the literature.     

Random Effects Misspecification Can Have Severe Consequences for Random Effects Inference in Linear Mixed Models

There has been considerable and controversial research over the past two decades into how successfully random effects misspecification in mixed models (i.e. assuming normality for the random effects when the true distribution is non‐normal) can be diagnosed and what its impacts are on estimation and inference. However, much of this research has focused on fixed effects inference in generalised linear mixed models. In this article, motivated by the increasing number of applications of mixed models where interest is on the variance components, we study the effects of random effects misspecification on random effects inference in linear mixed models, for which there is considerably less literature. Our findings are surprising and contrary to general belief: for point estimation, maximum likelihood estimation of the variance components under misspecification is consistent, although in finite samples, both the bias and mean squared error can be substantial. For inference, we show through theory and simulation that under misspecification, standard likelihood ratio tests of truly non‐zero variance components can suffer from severely inflated type I errors, and confidence intervals for the variance components can exhibit considerable under coverage. Furthermore, neither of these problems vanish asymptotically with increasing the number of clusters or cluster size. These results have major implications for random effects inference, especially if the true random effects distribution is heavier tailed than the normal. Fortunately, simple graphical and goodness‐of‐fit measures of the random effects predictions appear to have reasonable power at detecting misspecification. We apply linear mixed models to a survey of more than 4 000 high school students within 100 schools and analyse how mathematics achievement scores vary with student attributes and across different schools. The application demonstrates the sensitivity of mixed model inference to the true but unknown random effects distribution.     

A new family of bivariate max-infinitely divisible distributions

In this article we discuss the asymptotic behaviour of the componentwise maxima for a specific bivariate triangular array. Its components are given in terms of linear transformations of bivariate generalised symmetrised Dirichlet random vectors introduced in Fang and Fang (Statistical inference in elliptically contoured and related distributions. Allerton Press, New York, 1990). We show that the componentwise maxima of such triangular arrays is attracted by a bivariate max-infinitely divisible distribution function, provided that the associated random radius is in the Weibull max-domain of attraction.     

Maximum likelihood estimation of random effects models

Iterated GLS has a remarkable property when applied to the random effects model in its usual parameterization. The values for the parameter that measures relative variance, obtained through successive iterations, form a monotonic sequence. This property provides convenient checks for multiple maxima of the likelihood function and for existence of a local maximum that satisfies the non-negativity condition.     

On the inefficiency of the restricted maximum likelihood

The restricted maximum likelihood is preferred by many to the full maximum likelihood for estimation with variance component and other random coefficient models, because the variance estimator is unbiased. It is shown that this unbiasedness is accompanied in some balanced designs by an inflation of the mean squared error. An estimator of the cluster‐level variance that is uniformly more efficient than the full maximum likelihood is derived. Estimators of the variance ratio are also studied.     

Bayesian model selection using encompassing priors

This paper deals with Bayesian selection of models that can be specified using inequality constraints among the model parameters. The concept of encompassing priors is introduced, that is, a prior distribution for an unconstrained model from which the prior distributions of the constrained models can be derived. It is shown that the Bayes factor for the encompassing and a constrained model has a very nice interpretation: it is the ratio of the proportion of the prior and posterior distribution of the encompassing model in agreement with the constrained model. It is also shown that, for a specific class of models, selection based on encompassing priors will render a virtually objective selection procedure. The paper concludes with three illustrative examples: an analysis of variance with ordered means; a contingency table analysis with ordered odds-ratios; and a multilevel model with ordered slopes.     

Self-weighted and local quasi-maximum likelihood estimators for ARMA-GARCH/IGARCH models

The limit distribution of the quasi-maximum likelihood estimator (QMLE) for parameters in the ARMA-GARCH model remains an open problem when the process has infinite 4th moment. We propose a self-weighted QMLE and show that it is consistent and asymptotically normal under only a fractional moment condition. Based on this estimator, the asymptotic normality of the local QMLE is established for the ARMA model with GARCH (finite variance) and IGARCH errors. Using the self-weighted and the local QMLEs, we construct Wald statistics for testing linear restrictions on the parameters, and their limiting distributions are given. In addition, we show that the tail index of the IGARCH process is always 2, which is independently of interest.     

Natural conjugate priors for the instrumental variables regression model applied to the Angrist–Krueger data

We propose a natural conjugate prior for the instrumental variables regression model. The prior is a natural conjugate one since the marginal prior and posterior of the structural parameter have the same functional expressions which directly reveal the update from prior to posterior. The Jeffreys prior results from a specific setting of the prior parameters and results in a marginal posterior of the structural parameter that has an identical functional form as the sampling density of the limited information maximum likelihood estimator. We construct informative priors for the Angrist–Krueger [1991. Does compulsory school attendance affect schooling and earnings? Quarterly Journal of Economics 106, 979–1014] data and show that the marginal posterior of the return on education in the US coincides with the marginal posterior from the Southern region when we use the Jeffreys prior. This result occurs since the instruments are the strongest in the Southern region and the posterior using the Jeffreys prior, identical to maximum likelihood, focusses on the strongest available instruments. We construct informative priors for the other regions that make their posteriors of the return on education similar to that of the US and the Southern region. These priors show the amount of prior information needed to obtain comparable results for all regions.     

Comparison of Hypothesis Testing and Bayesian Model Selection

The main goal of both Bayesian model selection and classical hypotheses testing is to make inferences with respect to the state of affairs in a population of interest. The main differences between both approaches are the explicit use of prior information by Bayesians, and the explicit use of null distributions by the classicists. Formalization of prior information in prior distributions is often difficult. In this paper two practical approaches (encompassing priors and training data) to specify prior distributions will be presented. The computation of null distributions is relatively easy. However, as will be illustrated, a straightforward interpretation of the resulting p-values is not always easy. Bayesian model selection can be used to compute posterior probabilities for each of a number of competing models. This provides an alternative for the currently prevalent testing of hypotheses using p-values. Both approaches will be compared and illustrated using case studies. Each case study fits in the framework of the normal linear model, that is, analysis of variance and multiple regression.     

A local maximum likelihood estimator for Poisson regression

A local maximum likelihood estimator based on Poisson regression is presented as well as its bias, variance and asymptotic distribution. This semiparametric estimator is intended to be an alternative to the Poisson, negative binomial and zero-inflated Poisson regression models that does not depend on regularity conditions and model specification accuracy. Some simulation results are presented. The use of the local maximum likelihood procedure is illustrated on one example from the literature. This procedure is found to perform well. This research was partially supported by Calouste Gulbenkian Foundation and PRODEP III.     

Analysis of variance on indirectly measured variables

Abstract  When observations from a normal distribution can only be obtained indirectly by counting the number of subjects responding to a previously chosen dose, parameter estimates can be obtained by using probit analysis. Well-known is the maximum likelihood technique of parameter estimation, less known is the approach by weighted least squares. The latter approach is followed to compare the parameters of several normal distributions by testing their equality, in analogy with the analysis of variance. A practical situation gave rise to this study and it is worked out at the end of the paper.     

Reparameterized inverse gamma regression models with varying precision

In this article, we propose a mean linear regression model where the response variable is inverse gamma distributed using a new parameterization of this distribution that is indexed by mean and precision parameters. The main advantage of our new parametrization is the straightforward interpretation of the regression coefficients in terms of the expectation of the positive response variable, as usual in the context of generalized linear models. The variance function of the proposed model has a quadratic form. The inverse gamma distribution is a member of the exponential family of distributions and has some distributions commonly used for parametric models in survival analysis as special cases. We compare the proposed model to several alternatives and illustrate its advantages and usefulness. With a generalized linear model approach that takes advantage of exponential family properties, we discuss model estimation (by maximum likelihood), black further inferential quantities and diagnostic tools. A Monte Carlo experiment is conducted to evaluate the performances of these estimators in finite samples with a discussion of the obtained results. A real application using minerals data set collected by Department of Mines of the University of Atacama, Chile, is considered to demonstrate the practical potential of the proposed model.     

On model selection in Bayesian regression

We discuss the problem of constructing a suitable regression model from a nonparametric Bayesian viewpoint. For this purpose, we consider the case when the error terms have symmetric and unimodal densities. By the Khintchine and Shepp theorem, the density of response variable can be written as a scale mixture of uniform densities. The mixing distribution is assumed to have a Dirichlet process prior. We further consider appropriate prior distributions for other parameters as the components of the predictive device. Among the possible submodels, we select the one which has the highest posterior probability. An example is given to illustrate the approach.     

Bayesian analysis of a Tobit quantile regression model

This paper develops a Bayesian framework for Tobit quantile regression. Our approach is organized around a likelihood function that is based on the asymmetric Laplace distribution, a choice that turns out to be natural in this context. We discuss families of prior distributions on the quantile regression vector that lead to proper posterior distributions with finite moments. We show how the posterior distribution can be sampled and summarized by Markov chain Monte Carlo methods. A method for comparing alternative quantile regression models is also developed and illustrated. The techniques are illustrated with both simulated and real data. In particular, in an empirical comparison, our approach out-performed two other common classical estimators.     

Bayesian estimation of transition probabilities from repeated cross sections

This paper discusses some simple practical advantages of Markov chain Monte Carlo (MCMC) methods in estimating entry and exit transition probabilities from repeated independent surveys. Simulated data are used to illustrate the usefulness of MCMC methods when the likelihood function has multiple local maxima. Actual data on the evaluation of an HIV prevention intervention program among drug users are used to demonstrate the advantage of using prior information to enhance parameter identificaiton. The latter example also demonstrates an important strength of the MCMC approach, namely the ability to make inferences on arbitrary functions of model parameters.     

Stochastic volatility with leverage: Fast and efficient likelihood inference

This paper is concerned with the Bayesian analysis of stochastic volatility (SV) models with leverage. Specifically, the paper shows how the often used Kim et al. [1998. Stochastic volatility: likelihood inference and comparison with ARCH models. Review of Economic Studies 65, 361–393] method that was developed for SV models without leverage can be extended to models with leverage. The approach relies on the novel idea of approximating the joint distribution of the outcome and volatility innovations by a suitably constructed ten-component mixture of bivariate normal distributions. The resulting posterior distribution is summarized by MCMC methods and the small approximation error in working with the mixture approximation is corrected by a reweighting procedure. The overall procedure is fast and highly efficient. We illustrate the ideas on daily returns of the Tokyo Stock Price Index. Finally, extensions of the method are described for superposition models (where the log-volatility is made up of a linear combination of heterogenous and independent autoregressions) and heavy-tailed error distributions (student and log-normal).     

On the estimation of the parameters for the Littlewood model in software reliability

A very well-known model in software reliability theory is that of Littlewood (1980). The (three) parameters in this model are usually estimated by means of the maximum likelihood method. The system of likelihood equations can have more than one solution. Only one of them will be consistent, however. In this paper we present a different, more analytical approach, exploiting the mathematical properties of the log-likelihood function itself. Our belief is that the ideas and methods developed in this paper could also be of interest for statisticians working on the estimation of the parameters of the generalised Pareto distribution. For those more generally interested in maximum likelihood the paper provides a 'practical case', indicating how complex matters may become when only three parameters are involved. Moreover, readers not familiar with counting process theory and software reliability are given a first introduction.     

A simple approach for the analysis of generalizea linear mixed models

A broad class of generalized linear mixed models, e.g. variance components models for binary data, percentages or count data, will be introduced by incorporating additional random effects into the linear predictor of a generalized linear model structure. Parameters are estimated by a combination of quasi-likelihood and iterated MINQUE (minimum norm quadratic unbiased estimation), the latter being numerically equivalent to REML (restricted, or residual, maximum likelihood). First, conditional upon the additional random effects, observations on a working variable and weights are derived by quasi-likelihood, using iteratively re-weighted least squares. Second, a linear mixed model is fitted to the working variable, employing the weights for the residual error terms, by iterated MINQUE. The latter may be regarded as a least squares procedure applied to squared and product terms of error contrasts derived from the working variable. No full distributional assumptions are needed for estimation. The model may be fitted with standardly available software for weighted regression and REML.     

Linear and nonlinear regression with stable errors

In this paper we describe methods and evaluate programs for linear regression by maximum likelihood when the errors have a heavy tailed stable distribution. The asymptotic Fisher information matrix for both the regression coefficients and the error distribution parameters are derived, giving large sample confidence intervals for all parameters. Simulated examples are shown where the errors are stably distributed and also where the errors are heavy tailed but are not stable, as well as a real example using financial data. The results are then extended to nonlinear models and to non-homogeneous error terms.     

The assessment of the expected value of information in the binary decision model

The expected value of information represents the maximum amount the decision maker should spend on inquiry before making a decision. This amount depends upon the accuracy of the information. In many cases of inquiry, prior objective knowledge of the accuracy is not available. This paper presents and compares two methods of subjectively assessing the value of imperfect information in the binary decision model. In the first method, the decision maker provides a likelihood function for the inquiry and hence the probabilities of error. The second method is the preposterior approach, in which the decision maker provides the prior distribution for the posterior probability.