Statistical inference on restricted partial linear regression models with partial distortion measurement errors

We consider the estimation and hypothesis testing problems for the partial linear regression models when some variables are distorted with errors by some unknown functions of commonly observable confounding variable. The proposed estimation procedure is designed to accommodate undistorted as well as distorted variables. To test a hypothesis on the parametric components, a restricted least squares estimator is proposed under the null hypothesis. Asymptotic properties for the estimators are established. A test statistic based on the difference between the residual sums of squares under the null and alternative hypotheses is proposed, and we also obtain the asymptotic properties of the test statistic. A wild bootstrap procedure is proposed to calculate critical values. Simulation studies are conducted to demonstrate the performance of the proposed procedure, and a real example is analyzed for an illustration.     

Analysis of variance on indirectly measured variables

Abstract  When observations from a normal distribution can only be obtained indirectly by counting the number of subjects responding to a previously chosen dose, parameter estimates can be obtained by using probit analysis. Well-known is the maximum likelihood technique of parameter estimation, less known is the approach by weighted least squares. The latter approach is followed to compare the parameters of several normal distributions by testing their equality, in analogy with the analysis of variance. A practical situation gave rise to this study and it is worked out at the end of the paper.     

Multicollinearity in regression: an efficiency comparison between L<Subscript>p</Subscript>-norm and least squares estimators

Multicollinearity is one of the most important issues in regression analysis, as it produces unstable coefficients’ estimates and makes the standard errors severely inflated. The regression theory is based on specific assumptions concerning the set of error random variables. In particular, when errors are uncorrelated and have a constant variance, the ordinary least squares estimator produces the best estimates among all linear estimators. If, as often happens in reality, these assumptions are not met, other methods might give more efficient estimates and their use is therefore recommendable. In this paper, after reviewing and briefly describing the salient features of the methods, proposed in the literature, to determine and address the multicollinearity problem, we introduce the L_pmin method, based on L_p-norm estimation, an adaptive robust procedure that is used when the residual distribution has deviated from normality. The major advantage of this approach is that it produces more efficient estimates of the model parameters, for different degrees of multicollinearity, than those generated by the ordinary least squares method. A simulation study and a real-data application are also presented, in order to show the better results provided by the L_pmin method in the presence of multicollinearity.     

MSE-improvement of the least squares estimator by dropping variables

It is well known that dropping variables in regression analysis decreases the variance of the least squares (LS) estimator of the remaining parameters. However, after elimination estimates of these parameters are biased, if the full model is correct. In his recent paper, Boscher (1991) showed that the LS-estimator in the special case of a mean shift model (cf. Cook and Weisberg, 1982) which assumes no “outliers” can be considered in the framework of a linear regression model where some variables are deleted. He derived conditions under which this estimator outperforms the LS-estimator of the full model in terms of the mean squared error (MSE)-matrix criterion. We demonstrate that this approach can be extended to the general set-up of dropping variables. Necessary and sufficient conditions for the MSE-matrix superiority of the LS-estimator in the reduced model over that in the full model are derived. We also provide a uniformly most powerful F-statistic for testing the MSE-improvement.     

A new measurement procedure for attitudinal research. Analysis of its psychometric and informational properties

Since measurement errors have strong effects in all relationships (statistical or otherwise) studied, there is an increasing interest in the data quality, which is the major justification for this research. This paper aims to present a new measurement procedure, the letter scale, which avoids many of the problems connected with the response modalities traditionally used in attitudinal research, especially the ordinal categorical scales. This paper analyzes the error composition of the scores obtained with this new measurement procedure. The validity of the procedure is also analyzed and the observed variance is assessed to determine which part of the observed variance is “valid”, which part is random error (attenuating relationships) and which is correlated error (magnifying relationships). Structural equation models will be used to provide estimates of the measurement quality: (i) Reliability, (ii) Construct validity, method effect and residual variance. In addition, this letter scale is evaluated under another different perspective, Information Theory measures are also used to assess the amount of information transmitted. The relative merits of this new measurement procedure as opposed to other common response modalities will be discussed in both cases.     

Residential preferences, neighborhood filtering and neighborhood change : Comment and corrections

The purpose of this Comment is to correct the estimating technique used by Little in “Residential Preferences, Neighborhood Filtering and Neighborhood Change.” Little uses the factor load matrix rather than the factor score matrix in his computations of the implicit regression coefficients. We correct Little's estimates and also present additional results to compare his corrected results with principal component estimates and ordinary least squares.     

A simple approach for the analysis of generalizea linear mixed models

A broad class of generalized linear mixed models, e.g. variance components models for binary data, percentages or count data, will be introduced by incorporating additional random effects into the linear predictor of a generalized linear model structure. Parameters are estimated by a combination of quasi-likelihood and iterated MINQUE (minimum norm quadratic unbiased estimation), the latter being numerically equivalent to REML (restricted, or residual, maximum likelihood). First, conditional upon the additional random effects, observations on a working variable and weights are derived by quasi-likelihood, using iteratively re-weighted least squares. Second, a linear mixed model is fitted to the working variable, employing the weights for the residual error terms, by iterated MINQUE. The latter may be regarded as a least squares procedure applied to squared and product terms of error contrasts derived from the working variable. No full distributional assumptions are needed for estimation. The model may be fitted with standardly available software for weighted regression and REML.     

A generalization of Friedman's rank statistic

Abstract  In this paper a very natural generalization of the two-way analysis of variance rank statistic of F riedman is given. The general distribution-free test procedure based on this statistic for the effect of J treatments in a random block design can be applied in general two-way layouts without interactions and with different numbers of the continuous observations per cell provided the design scheme is connected. The asymptotic distribution under the null hypothesis of the test statistic is derived. A comparison with the method of m rankings of B enard and van E lteren is made. The disadvantage of B enard and van E lteren's test procedure is that the number of observations per block does influence the statistic twice, namely firstly by the number itself, as it should, and see ondly by the level of the ranks which will be different in different blocks if the numbers of observations per block are different. The proposed test statistic is not sensitive to differences in the levels of the ranks caused by the different numbers of observations per block. The test is derived from considerhg the K ruskal -W allis statistics per block.
Finally, the results of simulation experiments are given. The simulation is carried out for three designs and a number of normal location alternatives and gives some information about the power of the suggested test procedure. A comparison is made with B enard and van E lteren's test and with the classical analysis of variance technique. For some simple orthogonal designs the exact null distributions of B enard and van E lteren's test and the proposed test are compared.     

Assessing the presence of harmful collinearity and other forms of weak data through a test for signal-to-noise

Data weaknesses (such as collinearity) reduce the quality of least-squares estimates by inflating parameter variances. Standard regression diagnostics and statistical tests of hypothesis are unable to indicate such variance inflation and hence cannot detect data weaknesses. In this paper, then, we consider a different means for determining the presence of weak data based on a test for signal-to-noise in which the size of the parameter variance (noise) is assessed relative to the magnitude of the parameter (signal). This test is combined with other collinearity diagnostics to provide a test for the presence of harmful collinearity and/or short data. The entire procedure is illustrated with an equation from the Michigan Quarterly Econometric Model. Tables of critical values for the test are provided in an appendix.     

Parametric Robust Test for Several Variances with Unknown Underlying Distributions

Tsou (in comm Stat-Theor Math 32: 2013–2019, 2003) proposed a parametric robust procedure for testing the equality of two population variances. With large samples the proposed test remains valid under model misspecification. In this article the robust technique is further extended to the comparison of several population variances. More specifically the score test derived on the basis of normal models is adjusted to become robust. The adjusted robust test provides asymptotically valid inference so long as the true underlying distributions have finite fourth moments. Unlike most robust nonparametric approaches, this novel robust technique too provides legitimate variance estimates for estimators of the interested parameters.     

Estimating panel data models in the presence of endogeneity and selection

We consider estimation of panel data models with sample selection when the equation of interest contains endogenous explanatory variables as well as unobserved heterogeneity. Assuming that appropriate instruments are available, we propose several tests for selection bias and two estimation procedures that correct for selection in the presence of endogenous regressors. The tests are based on the fixed effects two-stage least squares estimator, thereby permitting arbitrary correlation between unobserved heterogeneity and explanatory variables. The first correction procedure is parametric and is valid under the assumption that the errors in the selection equation are normally distributed. The second procedure estimates the model parameters semiparametrically using series estimators. In the proposed testing and correction procedures, the error terms may be heterogeneously distributed and serially dependent in both selection and primary equations. Because these methods allow for a rather flexible structure of the error variance and do not impose any nonstandard assumptions on the conditional distributions of explanatory variables, they provide a useful alternative to the existing approaches presented in the literature.     

Least squares and stochastic difference equations

In this paper we study the asymptotic properties of least squares estimates of parameters in a stochastic difference equation. The difference equation is assumed to be linear with constant real coefficients. Moreover, the roots of the associated characteristic polynomial are all assumed to have absolute value different from one. Finally, the difference equation is assumed to be non-homogeneous with fixed (i.e., non-random) initial values, and to have ‘error terms’ that are independently and identically distributed with mean zero and finite variance. We show that under these conditions the least squares estimates of the coefficients of the difference equation converge with probability one to the true values.     

Adaptive estimation of autoregressive models with time-varying variances

Stable autoregressive models are considered with martingale differences errors scaled by an unknown nonparametric time-varying function generating heterogeneity. An important special case involves structural change in the error variance, but in most practical cases the pattern of variance change over time is unknown and may involve shifts at unknown discrete points in time, continuous evolution or combinations of the two. This paper develops kernel-based estimators of the residual variances and associated adaptive least squares (ALS) estimators of the autoregressive coefficients. Simulations show that efficiency gains are achieved by the adaptive procedure.     

Canonical forms and tests of hypotheses Part II: Multivariate mixed linear models

Abstract  In the literature on multivariate analysis of variance, exact test procedures are restricted to linear models with fixed effects only. In this paper tests are presented for multivarite linear hypotheses with respect to mixed models, which constitude a generalization of (univariate) regular models described by R oebruck (1982). Furthermore it is shown, that the matrices, which are used to compute the test statistics, can be derived from the univariate "sums of squares" in the same manner as in the case of fixed models. The applicability of this theory is demonstrated by two examples.     

Nonlinearity measures: a case study

Summary An important problem in applied statistics is fitting a given model function f (β) with unknown parameters β to a data vector y. Minimizing the residual sum of squares provides the least squares estimates of β. If f (β) is linear in β the precision of these estimates is well-known. In a nonlinear case approximate (though asymptotically exact) confidence statements can be made. B eale [1] introduced measures of nonlinearity which can be used to indicate when approximate confidence statements are appropriate. G uttman and M eeter [2] showed that in some, severely nonlinear, cases Beale's measures do not give the right indication. In this paper two new nonlinearity measures are introduced and their use is illustrated on a practical problem described by W itt [3]. A more detailed discussion of the theoretical background can be found in references [1] and [2].     

Robust estimation for structural spurious regressions and a Hausman-type cointegration test

This paper analyzes an approach to correcting spurious regressions involving unit-root nonstationary variables by generalized least squares (GLS) using asymptotic theory. This analysis leads to a new robust estimator and a new test for dynamic regressions. The robust estimator is consistent for structural parameters not just when the regression error is stationary but also when it is unit-root nonstationary under certain conditions. We also develop a Hausman-type test for the null hypothesis of cointegration for dynamic ordinary least squares (OLS) estimation. We demonstrate our estimation and testing methods in three applications: (i) long-run money demand in the U.S., (ii) output convergence among industrial and developing countries, and (iii) purchasing power parity (PPP) for traded and non-traded goods.     

The incompatibility of achieving a fully specified linear model with assuming that residual dependent-variable variance is random

The residual dependent-variable variance in experiments is not “random error”, as it is often assumed to be, but merely “unaccounted for variance”, because what is random is inexplicable in terms of any possible set of independent-variables and this is something that ultimately is only empirically determinable. So, if there is any unaccounted for dependent-variable variance, an experiment’s set of independent-variables is certainly under-specified and perhaps mis-specified because of the confounding of variables included in this set by causally relevant variables not included in the set. Thus, the proper first empirical test of any linear model is whether it leaves any residual dependent-variable variance, and if it does then none of its independent variables can yet logically justifiably be claimed to predict or causally explain any of the dependent-variable variance whatsoever.     

Nested analysis of variance: Review of methods

Summary Methods of the nested analysis of variance procedures are discussed and a formalized three-way nested model is outlined. A summary of the statistical tests for the assumptions relevant to the model is given together with an outline of the numerical methods for calculating the sum of squares, the degrees of freedom, the mean squares, the tests of significance, and the estimate of variance components. Several methods are given for obtaining confidence limits for variance components. Procedures are presented for utilizing the variance components to estimate process and/or control limits and to obtain the most efficient or optimum sampling scheme. Some differences between the nested analysis of variance model and the crossed analysis of variance model are explained.     

Least squares learning and the US Treasury bill rate

Understanding how agents formulate their expectations about Fed behavior is important for market participants because they can potentially use this information to make more accurate estimates of stock and bond prices. Although it is commonly assumed that agents learn over time, there is scant empirical evidence in support of this assumption. Thus, in this paper we test if the forecast of the three month T-bill rate in the Survey of Professional Forecasters (SPF) is consistent with least squares learning when there are discrete shifts in monetary policy. We first derive the mean, variance and autocovariances of the forecast errors from a recursive least squares learning algorithm when there are breaks in the structure of the model. We then apply the Bai and Perron (1998) test for structural change to a forecasting model for the three month T-bill rate in order to identify changes in monetary policy. Having identified the policy regimes, we then estimate the implied biases in the interest rate forecasts within each regime. We find that when the forecast errors from the SPF are corrected for the biases due to shifts in policy, the forecasts are consistent with least squares learning.     

Return to education in Finland

This study presents estimates of the return to education in Finland using an individual-level data set that also includes ability measures and information on family background. It is found that ability test scores have a strong effect on the choice of education and on subsequent earnings. Estimating the return to education with no information on ability leads to an upward bias in the estimates. However, this bias is more than offset by a downward bias caused by endogeneity or measurement error. Instrumental variables estimates that utilize family background variables as instruments produce estimates of the return to schooling that are approximately 60% higher than the least squares estimates.