Variance estimation in the change analysis of a linear regression model

We study a “direct test” of Chu and White (1992) proposed for detecting changes in the trend of a linear regression model. The power of this test strongly depends on a suitable estimation of the variance of the error variables involved. We discuss various types of variance estimators and derive their asymptotic properties under the null-hypothesis of “no change” as well as under the alternative of “a change in linear trend”. A small simulation study illustrates the estimators' finite sample behaviour.     

The Asymptotics of MM-Estimators for Linear Regression with Fixed Designs

MM-estimators achieve simultaneous high efficiency and high breakdown point over contamination neighborhoods. Inference based on these estimators relies on their asymptotic properties, which have been studied for the case of random covariates. In this paper we show that, under relatively mild regularity conditions, MM-estimators for linear regression models are strongly consistent when the design is fixed. Moreover, their strong consistency allows us to show that these estimators are also asymptotically normal for non-random covariates. These results justify the use of a normal approximation to the finite-sample distribution of MM-estimators for linear regression with fixed explanatory variables. Additionally, these results have been used to extend the robust bootstrap (Salibian-Barrera and Zamar in Ann Stat 30:556–582, 2002) to the case of fixed designs [see Salibian-Barrera 2004, submitted].Research supported by an NSERC Research Grant (Individual)     

Estimating regression coefficients from survey data by asymptotic design-cum-model based approach

Postulating a linear regression of a variable of interest on an auxiliary variable with values of the latter known for all units of a survey population, we consider appropriate ways of choosing a sample and estimating the regression parameters. Recalling Thomsen’s (1978) results on non-existence of ‘design-cum-model’ based minimum variance unbiased estimators of regression coefficients we apply Brewer’s (1979) ‘asymptotic’ analysis to derive ‘asymptotic-design-cummodel’ based optimal estimators assuming large population and sample sizes. A variance estimation procedure is also proposed.     

Preliminary test estimators of the parameters of simple linear model with measurement error

In this paper, we discuss the properties of preliminary test estimators (PTE) of the parameters of simple linear model with measurement error (ME model) when the slope of the linear model is suspected to be zero. Expressions of the bias, MSE and efficiencies are obtained under conditional as well as unconditional situations with known reliability coefficient. Conditional model results are compared to the standard model without measurement error. We also provide the unconditional model analysis in finite samples. Asymptotic theory under local alternatives is developed when the variance of measurement error or the ratio of the variance of the model error relative to the variance of the measurement error is known. Asymptotic expressions of bias and MSE of the estimators along with their efficiencies are obtained. In every case, it is shown that the measurement error tend to increase the variability of the estimators compared to the estimators without measurement error. Graphs and tables are provided to see these results and to determine optimum level of significance for minimum guaranteed efficiency. Received October 2001 RID="*" ID="*"  A. K. Md. E. Saleh is a Distinguished Research Professor and H. M. Kim is a Ph.D. candidate in the School of Mathematics and Statistics, Carleton University, Ottawa. Acknowledgment. The authors gratefully acknowledge the constructive suggestion of the referees to improve the paper. The research is supported by NSERC grant A3088.     

Estimation of the parameters of log-gamma distribution using order statistics

In this work we propose a technique of estimating the location parameter μ and scale parameter σ of log-gamma distribution by U-statistics constructed by taking best linear functions of order statistics as kernels. The efficiency comparison of the proposed estimators with respect to maximum likelihood estimators is also made.     

Estimators of Regression Parameters for Truncated and Censored Data

Estimators of parameters in semi-parametric left truncated and right censored regression models are proposed. In contrast to the majority of existing estimators, the proposed estimators do not require the error term of the regression model to have a symmetric distribution. In addition the estimators use asymmetric “trimming” of observations. Consistency and asymptotic normality of the estimators are shown. Finite sample properties are considered in a small simulation study. For the left truncated case, an empirical application illustrates the usefulness of the estimator.     

On equalities of estimations of parametric functions under a general linear model and its restricted models

Estimations of parametric functions under a general linear model and its restricted models involve some complicated operations of matrices and their generalized inverses. In the past several years, a powerful tool—the matrix rank method was utilized to manipulate various complicated matrix expressions that involve generalized inverses of matrices. In this paper, we use this method to derive necessary and sufficient conditions for six equalities of the ordinary least-squares estimators and the best linear unbiased estimators of parametric functions to equal under a general linear model and its corresponding restricted model.     

Survey statisticians celebrate golden jubilee year 2003 of the linear regression estimator

In the present investigation, a general set-up for inference from survey data that covers the estimation of variance of estimators of totals and distribution functions has been considered, using known first and second order moments of auxiliary information at the estimation stage. The traditional linear regression estimator of population total owed to Hansen et al. Sample survey methods and theory. vol. 1 & 2, New York, Wiley (1953) is shown to be unique in its class of estimators, and celebrates Golden Jubilee Year-2003 for its outstanding performance in the literature by following Singh Advanced sampling theory with applications: How Michael selected Amy, vols 1 & 2, Kluwer, The Netherlands, pp 1–1247 2003. This particular paper has been designed to repair the methodology of Rao J. Off Stat 10(2):153–165 (1994) and hence that of Singh Ann Ins Stat Math 53(2):404–417 (2001). Although there is no need of simulation study to demonstrate the superiority of the proposed technique, because the theoretical results are crystal clear, but a small scale level simulation study have been designed to show the performance of the proposed estimators over the existing estimators in the literature.     

Estimation of multiple poisson means: A compromise between maximum likelihood and empirical bayes estimators

For estimatingp(⩾ 2) independent Poisson means, the paper considers a compromise between maximum likelihood and empirical Bayes estimators. Such compromise estimators enjoy both good componentwise as well as ensemble properties. Research supported by the NSF Grant Number MCS-8218091.     

Bayesian estimation and prediction based on Rayleigh sample quantiles

Ordered data arise naturally in many fields of statistical practice. Often some sample values are unknown or disregarded due to various reasons. On the basis of some sample quantiles from the Rayleigh distribution, the problems of estimating the Rayleigh parameter, hazard rate and reliability function, and predicting future observations are addressed using a Bayesian perspective. The construction of β-content and β-expectation Bayes tolerance limits is also tackled. Under squared-error loss, Bayes estimators and predictors are deduced analytically. Exact tolerance limits are derived by solving simple nonlinear equations. Highest posterior density estimators and credibility intervals, as well as Bayes estimators and predictors under linear loss, can easily be computed iteratively.     

The second-order bias and mean squared error of estimators in time-series models

We develop analytical results on the second-order bias and mean squared error of estimators in time-series models. These results provide a unified approach to developing the properties of a large class of estimators in linear and nonlinear time-series models and they are valid for both normal and nonnormal samples of observations, and where the regressors are stochastic. The estimators included are the generalized method of moments, maximum likelihood, least squares, and other extremum estimators. Our general results are applied to four time-series models. We investigate the effects of nonnormality on the second-order bias results for two of these models, while for all four models, the second-order bias and mean squared error results are given under normality. Numerical results for some of these models are also presented.     

Mean square error estimation in multi-stage sampling

Summary: Suppose for a homogeneous linear unbiased function of the sampled first stage unit (fsu)-values taken as an estimator of a survey population total, the sampling variance is expressed as a homogeneous quadratic function of the fsu-values. When the fsu-values are not ascertainable but unbiased estimators for them are separately available through sampling in later stages and substituted into the estimator, Raj (1968) gave a simple variance estimator formula for this multi-stage estimator of the population total. He requires that the variances of the estimated fsu-values in sampling at later stages and their unbiased estimators are available in certain `simple forms'. For the same set-up Rao (1975) derived an alternative variance estimator when the later stage sampling variances have more ‘complex forms’. Here we pursue with Raj's (1968) simple forms to derive a few alternative variance and mean square error estimators when the condition of homogeneity or unbiasedness in the original estimator of the total is relaxed and the variance of the original estimator is not expressed as a quadratic form.  We illustrate a particular three-stage sampling strategy and present a simulation-based numerical exercise showing the relative efficacies of two alternative variance estimators. Received: 19 February 1999     

Riesz estimators

We consider properties of estimators that can be written as vector lattice (Riesz space) operations. Using techniques widely used in economic theory and functional analysis, we study the approximation properties of these estimators paying special attention to additive models. We also provide two algorithms RIESZVAR(i-ii) for the consistent parametric estimation of continuous multivariate piecewise linear functions.     

Second-order properties of estimators of serial correlation from regression residuals

Second-order properties of estimators and tests offer a way of choosinf among aymptotically equivalent procedures. This paper studies the second-order terms of two estimators of serial correlation in the linear model. Using these second-order approximations, the maximum likelihood estimator is judge to be superior in terms of bias and variance. A small Monte Carlo experiment is done to assess the accuracy of the results.     

Approximate inference in multivariate nonlinear functional relationships

An estimation procedure based on estimating equations is presented for the parameters in a multivariate functional relationship model, where all observations are subject to error. The covariance matrix of the observational errors may be parametrized and is allowed to be different for different sets of observations. Estimators are defined for the unknown relation parameters and error parameters.
For linear models (i.e. where the model function is linear in the incidental parameters) the estimators are consistent and asymptotically normal. A consistent expression for the covariance matrix of the estimators is derived. The results are valid for general error distributions.
For nonlinear models the estimators are based on locally linear approximations to the model function. The afore mentioned properties of the estimators are now only approximately valid. The adequacy of the approximate inference, based on asymptotic theory for the linearized model, needs at least informal check. Some examples are given to illustrate the estimation procedure.     

On kernel nonparametric regression designed for complex survey data

In this article, we consider nonparametric regression analysis between two variables when data are sampled through a complex survey. While nonparametric regression analysis has been widely used with data that may be assumed to be generated from independently and identically distributed (iid) random variables, the methods and asymptotic analyses established for iid data need to be extended in the framework of complex survey designs. Local polynomial regression estimators are studied, which include as particular cases design-based versions of the Nadaraya–Watson estimator and of the local linear regression estimator. In this paper, special emphasis is given to the local linear regression estimator. Our estimators incorporate both the sampling weights and the kernel weights. We derive the asymptotic mean squared error (MSE) of the kernel estimators using a combined inference framework, and as a corollary consistency of the estimators is deduced. Selection of a bandwidth is necessary for the resulting estimators; an optimal bandwidth can be determined, according to the MSE criterion in the combined mode of inference. Simulation experiments are conducted to illustrate the proposed methodology and an application with the Canadian survey of labour and income dynamics is presented.     

Model reference adaptive systems applied to regression analyses

The adaptive estimation procedure of model reference adaptive systems is modified and applied to linear models. In general the principle can be used for almost any time series model. Because of the recursive nature of the resulting estimator, it is computationally appealing, especially when a time series is considered as a flow of data. In addition, the estimator turns out to have certain statistical optimality properties.
In the linear regression setting, Ridge estimators turn out to constitute a subclass of the adaptive estimators considered, whereas for unknown measurement variance, the resulting estimators are related to J ames -S tkin type estimators, and have better properties than the latter. The estimator is shown to be strongly consistent and to converge in law to a normal variate under the standard assumptions of linear models. Further it is shown to be admissible and minimax in restricted parameter spaces. The connection between K alman filters and the classical least-squares estimator is also pointed out.     

On MLEs in an extended multivariate linear growth curve model

In this paper the extended growth curve model is considered. The literature comprises two versions of the model. These models can be connected by one-to-one reparameterizations but since estimators are non-linear it is not obvious how to transmit properties of estimators from one model to another. Since it is only for one of the models where detailed knowledge concerning estimators is available (Kollo and von Rosen, Advanced multivariate statistics with matrices. Springer, Dordrecht, 2005) the object in this paper is therefore to present uniqueness properties and moment relations for the estimators of the second model. One aim of the paper is also to complete the results for the model presented in Kollo and von Rosen (Advanced multivariate statistics with matrices. Springer, Dordrecht, 2005). The presented proofs of uniqueness for linear combinations of estimators are valid for both models and are simplifications of proofs given in Kollo and von Rosen (Advanced multivariate statistics with matrices. Springer, Dordrecht, 2005).     

Estimating the autocorrelated error model with trended data

A Monte Carlo study of the small sample properties of various estimators of the linear regression model with first-order autocorrelated errors. When independent variables are trended, estimators using Ttransformed observations (Prais-Winsten) are much more efficient than those using T–1 (Cochrane–Orcutt). The best of the feasible estimators isiterated Prais-Winsten using a sum-of-squared-error minimizing estimate of the autocorrelation coefficient ?. None of the feasible estimators performs well in hypothesis testing; all seriously underestimate standard errors, making estimated coefficients appear to be much more significant than they actually are.     

Nonparametric estimation of conditional VaR and expected shortfall

This paper considers a new nonparametric estimation of conditional value-at-risk and expected shortfall functions. Conditional value-at-risk is estimated by inverting the weighted double kernel local linear estimate of the conditional distribution function. The nonparametric estimator of conditional expected shortfall is constructed by a plugging-in method. Both the asymptotic normality and consistency of the proposed nonparametric estimators are established at both boundary and interior points for time series data. We show that the weighted double kernel local linear conditional distribution estimator has the advantages of always being a distribution, continuous, and differentiable, besides the good properties from both the double kernel local linear and weighted Nadaraya–Watson estimators. Moreover, an ad hoc data-driven fashion bandwidth selection method is proposed, based on the nonparametric version of the Akaike information criterion. Finally, an empirical study is carried out to illustrate the finite sample performance of the proposed estimators.