A simple approach for the analysis of generalizea linear mixed models

A broad class of generalized linear mixed models, e.g. variance components models for binary data, percentages or count data, will be introduced by incorporating additional random effects into the linear predictor of a generalized linear model structure. Parameters are estimated by a combination of quasi-likelihood and iterated MINQUE (minimum norm quadratic unbiased estimation), the latter being numerically equivalent to REML (restricted, or residual, maximum likelihood). First, conditional upon the additional random effects, observations on a working variable and weights are derived by quasi-likelihood, using iteratively re-weighted least squares. Second, a linear mixed model is fitted to the working variable, employing the weights for the residual error terms, by iterated MINQUE. The latter may be regarded as a least squares procedure applied to squared and product terms of error contrasts derived from the working variable. No full distributional assumptions are needed for estimation. The model may be fitted with standardly available software for weighted regression and REML.     

On random fuzzy variables of second order and their application to linear statistical inference with fuzzy data

This paper summarizes some results on random fuzzy variables with existing expectation and variance, called random fuzzy variables of second order. Using the Frechét-principle and – via support functions – the embedding of convex fuzzy sets into a Banach space of functions it especially presents a unified view on expectation and variance of random fuzzy variables. These notions are applied in developing linear statistical inference with fuzzy data. Detailed investigations are presented concerning best linear unbiased estimation in linear regression models with fuzzy observations. Received: November 1999     

Some statistical problems in the presence of an outlier when sampling from truncation parameter densities

Summary In this paper we derive some recurrence relations for moments of order statistics of a random sample from a truncation parameter density when one of the observations is an outlier. We also derive uniform minimum variance unbiased estimator of a parametric function.     

Simultaneous optimal estimation in linear mixed models

Simultaneous optimal estimation in linear mixed models is considered. A necessary and sufficient condition is presented for the least squares estimator of the fixed effects and the analysis of variance estimator of the variance components to be of uniformly minimum variance simultaneously in a general variance components model. That is, the matrix obtained by orthogonally projecting the covariance matrix onto the orthogonal complement space of the column space of the design matrix is symmetric, each eigenvalue of the matrix is a linear combinations of the variance components and the number of all distinct eigenvalues of the matrix is equal to the the number of the variance components. Under this condition, uniformly optimal unbiased tests and uniformly most accurate unbiased confidence intervals are constructed for the parameters of interest. A necessary and sufficient condition is also given for the equivalence of several common estimators of variance components. Two examples of their application are given.     

Analysis and short-term predictions of non-technical loss of electric power based on mixed effects models

In this paper we estimate, analyze and predict short-term non-technical loss (NTL) of electric power of Brazilian energy service companies based on different assumptions for the covariance structure of the errors and controlling for socio-economic confounding variables. Although the correlation among repeated responses is not usually of intrinsic interest, it is an important aspect of the data that must properly be accounted for to produce valid inferences in longitudinal or panel data analysis. In the extended linear mixed effects model, the covariance matrix of the response vector is comprised by two subcomponents, a random effect component that can represent between group variation and a intraclass or within group component. So, in order to adequately treat the longitudinal character of NTL data, we use the decomposition of these variance components to evaluate different architectures to the within group errors. Using data of 59 Brazilian distributing utilities from 2004 to 2012, we fit a conditionally independent errors model and three other models with autoregressive-moving average parametrization to the intraclass disturbances. Finally, we compare models using the MAD and MAPE metrics in the prediction of NTL for the year of 2013. The findings suggest that the approach can be satisfactorily implemented in future statistical analysis of NTL.     

Mean square error estimation in multi-stage sampling

Summary: Suppose for a homogeneous linear unbiased function of the sampled first stage unit (fsu)-values taken as an estimator of a survey population total, the sampling variance is expressed as a homogeneous quadratic function of the fsu-values. When the fsu-values are not ascertainable but unbiased estimators for them are separately available through sampling in later stages and substituted into the estimator, Raj (1968) gave a simple variance estimator formula for this multi-stage estimator of the population total. He requires that the variances of the estimated fsu-values in sampling at later stages and their unbiased estimators are available in certain `simple forms'. For the same set-up Rao (1975) derived an alternative variance estimator when the later stage sampling variances have more ‘complex forms’. Here we pursue with Raj's (1968) simple forms to derive a few alternative variance and mean square error estimators when the condition of homogeneity or unbiasedness in the original estimator of the total is relaxed and the variance of the original estimator is not expressed as a quadratic form.  We illustrate a particular three-stage sampling strategy and present a simulation-based numerical exercise showing the relative efficacies of two alternative variance estimators. Received: 19 February 1999     

Admissible unbiased variance estimation in finite population sampling under randomized response

LetP be the proportion of units in a finite population possessing a sensitive attribute. We prove the admissibility of (i) an unbiased estimator of the variance of a general homogeneous linear unbiased estimator ofP and (ii) an unbiased estimator of the population varianceP(1−P), based on an arbitrary but fixed sampling design, under the randomized response plans due to Warner (1965) and Eriksson (1973). Admissibility of an unbiased strategy for estimating the population variance is also established.     

Approximate inference in multivariate nonlinear functional relationships

An estimation procedure based on estimating equations is presented for the parameters in a multivariate functional relationship model, where all observations are subject to error. The covariance matrix of the observational errors may be parametrized and is allowed to be different for different sets of observations. Estimators are defined for the unknown relation parameters and error parameters.
For linear models (i.e. where the model function is linear in the incidental parameters) the estimators are consistent and asymptotically normal. A consistent expression for the covariance matrix of the estimators is derived. The results are valid for general error distributions.
For nonlinear models the estimators are based on locally linear approximations to the model function. The afore mentioned properties of the estimators are now only approximately valid. The adequacy of the approximate inference, based on asymptotic theory for the linearized model, needs at least informal check. Some examples are given to illustrate the estimation procedure.     

A bias in estimating urban population density functions

This paper demonstrates that because of the rules used to delineate census tracts, unweighted estimation of an urban population density function using census tract observations leads to a severe upward bias in the estimated function. A weighted estimation procedure which leads to an unbiased estimate is proposed. The paper also points out that if one computes the integral of an unbiased estimate of a density function over the area of a city, that integral is not necessarily an unbiased estimate of total population. The paper thus explains the “disturbing” empirical results concerning density functions reported by McDonald and Bowman.     

Estimation of linear models with crossed-error structure

Sufficient conditions are presented under which the generalized least-squares estimator, with estimated covariance matrix, is unbiased for the parameters in the crossed-error model and has the same asymptotic distribution as the generalized least-squares estimator. The model permits the presence of independent variables that are constant over cross sections or time periods. The model does not require that the variance components associated with cross sections or time periods be positive.     

An overview of variance component estimation

Variance components estimation originated with estimating error variance in analysis of variance by equating error mean square to its expected value. This equating procedure was then extended to random effects models, first for balanced data (for which minimum variance properties were subsequently established) and later for unbalanced data. Unfortunately, this ANOVA methodology yields no optimum properties (other than unbiasedness) for estimation from unbalanced data. Today it is being replaced by maximum likelihood (ML) and restricted maximum likelihood (REML) based on normality assumptions and involving nonlinear equations that have to be solved numerically. There is also minimum norm quadratic unbiased estimation (MINQUE) which is closely related to REML but with fewer advantages.An invited paper for the ProbaStat '94 conference, Smolenice, Slovakia, May 30–June 3, 1994 Paper number BU-677 in the Biometrics Unit. Cornell University Ithaca NY     

Necessary and sufficient conditions for consistency of generalizedM-estimates

GeneralizedM-estimates (minimum contrast estimates) and their asymptotically equivalent approximate versions are considered. A relatively simple condition is found which is equivalent with consistency of all approximateM-estimates under wide assumptions about the model. This condition is applied in several directions. (i) A more easily verifiable condition equivalent with consistency of all approximateM-estimates is derived and illustrated on models with stationary and ergodic observations. (ii) A condition sufficient for inconsistency of all approximateM-estimates is obtained and illustrated on models with i.i.d. observations. (iii) A simple necessary and sufficient condition for consistency of all approximateM-estimates in linear regression with i.i.d. errors is found. This condition is weaker than sufficient conditions for consistency ofM-estimators known from the literature. A linear regression example is presented where theM-estimate is consistent and an approximateM-estimate is incosistent.Supported by CSAS grant N. 17503.     

A comparison between PPSWR and chaudhuri’s IPPS procedures

Chaudhuri (1975) suggested a simple procedure for extending any IPPS procedure involving two draws to an IPPS procedure for a general sample size, provided the size measures satisfy a certain condition. It is proved that the variance of the HTE based on Chaudhuri’s procedure is smaller than the variance of the usual unbiased estimator based on a PPSWR sample in the same number of draws, if the same is true for the procedure to start with.     

Estimating regression coefficients from survey data by asymptotic design-cum-model based approach

Postulating a linear regression of a variable of interest on an auxiliary variable with values of the latter known for all units of a survey population, we consider appropriate ways of choosing a sample and estimating the regression parameters. Recalling Thomsen’s (1978) results on non-existence of ‘design-cum-model’ based minimum variance unbiased estimators of regression coefficients we apply Brewer’s (1979) ‘asymptotic’ analysis to derive ‘asymptotic-design-cummodel’ based optimal estimators assuming large population and sample sizes. A variance estimation procedure is also proposed.     

Estimation in Two-Stage Models with Heteroscedasticity

A surprising number of important problems can be cast in the framework of estimating a mean and variance using data arising from a two-stage structure. The first stage is a random sampling of "units" with some quantity of interest associated with the unit. The second stage produces an estimate of that quantity and usually, but not always, an estimated standard error, which may change considerably across units. Heteroscedasticity in the estimates over different units can arise for a number of reasons, including variation associated with the unit and changing sampling effort over units. This paper presents a broad discussion of the problem of making inferences for the population mean and variance associated with the unobserved true values at the first stage of sampling. A careful discussion of the causes of heteroscedasticity is given, followed by an examination of ways in which inferences can be carried out in a manner that is robust to the nature of the within unit heteroscedasticity. Among the conclusions are that under any type of heteroscedasticity, an unbiased estimate of the mean and the variance of the estimated mean can be obtained by using the estimates as if they were true unobserved values from the first stage. The issue of using the mean versus a weighted average which tries to account for the heteroscedasticity is also discussed. An unbiased estimate of the population variance is given and the variance of this estimate and its covariance with the estimated mean is provided under various types of heteroscedasticity. The two-stage setting arises in many contexts including the one-way random effects models with replication, meta-analysis, multi-stage sampling from finite populations and random coefficients models. We will motivate and illustrate the problem with data arising from these various contexts with the goal of providing a unified framework for addressing such problems.     

Almost unbiased estimation in multiplicative models

In the lognormal linear models the estimation of constant term presents problems. In this paper we use weighted jackknife procedure (suggested by Hinkley 1977) for reducing the bias of the maximum likelihood estimator. The resulting estimator is unbiased upto order (1/T),T being the number of observations, and has the same MSE as that of the MLE to the same order of approximation; moreover, being the jackknife estimator it enjoys all the desirable large sample properties like any other jackknife estimator. The research of this author is partially supported through a research grant from NSERC of Canada.     

Fisher information under Gaussian quadrature models

This paper develops formulae to compute the Fisher information matrix for the regression parameters of generalized linear models with Gaussian random effects. The Fisher information matrix relies on the estimation of the response variance under the model assumptions. We propose two approaches to estimate the response variance: the first is based on an analytic formula (or a Taylor expansion for cases where we cannot obtain the closed form), and the second is an empirical approximation using the model estimates via the expectation–maximization process. Further, simulations under several response distributions and a real data application involving a factorial experiment are presented and discussed. In terms of standard errors and coverage probabilities for model parameters, the proposed methods turn out to behave more reliably than does the ‘disparity rule’ or direct extraction of results from the generalized linear model fitted in the last expectation–maximization iteration.     

The specification of least informative error distributions

Maximum likelihood estimation and most statistical tests require a full specification of the error distribution in a model. Under suitable parametric restrictions we can derive least informative specifications. The autoregressive processes prove to be least informative under a few simple variance and covariance restrictions. For the singular multivariate error distribution in a sum-constrained model, several least informative error distributions are obtained using different parametric assumptions on the covariance structure. A combined maximum entropy — maximum likelihood approach provides an alternative to other recent proposals for covariance estimation in small samples.     

Estimation and optimal designs for linear Haar-wavelet models

This paper gives an analytical expression for the best linear unbiased estimator (BLUE) of the unknown parameters in the linear Haar-wavelet model. From the analytical expression, we solve for the eigenvalues of the covariance matrix of the BLUE in analytical form. Further, we use these eigenvalues to construct some conventional discrete optimal designs for the model. The equivalences among these optimal designs are demonstrated and some examples are also given.