Nonparametric regression function estimation using interaction least squares splines and comlexity regularization

Let (X, Y) be a pair of random variables withsupp(X)⊆[0,1] ^l andEY ²<∞. Letm ^* be the best approximation of the regression function of (X, Y) by sums of functions of at mostd variables (1≤d≤l). Estimation ofm ^* from i.i.d. data is considered. For the estimation interaction least squares splines, which are defined as sums of polynomial tensor product splines of at mostd variables, are used. The knot sequences of the tensor product splines are chosen equidistant. Complexity regularization is used to choose the number of the knots and the degree of the splines automatically using only the given data. Without any additional condition on the distribution of (X, Y) the weak and strongL ₂-consistency of the estimate is shown. Furthermore, for everyp≥1 and every distribution of (X, Y) withsupp(X)⊆[0,1] ^l ,y bounded andm ^* p-smooth, the integrated squared error of the estimate achieves up to a logarithmic factor the (optimal) rate      

Consistency and asymptotic normality of least squares estimators in generalized STAR models

Space–time autoregressive (STAR) models, introduced by Cliff and Ord [Spatial autocorrelation (1973) Pioneer, London] are successfully applied in many areas of science, particularly when there is prior information about spatial dependence. These models have significantly fewer parameters than vector autoregressive models, where all information about spatial and time dependence is deduced from the data. A more flexible class of models, generalized STAR models, has been introduced in Borovkova et al. [Proc. 17th Int. Workshop Stat. Model. (2002), Chania, Greece] where the model parameters are allowed to vary per location. This paper establishes strong consistency and asymptotic normality of the least squares estimator in generalized STAR models. These results are obtained under minimal conditions on the sequence of innovations, which are assumed to form a martingale difference array. We investigate the quality of the normal approximation for finite samples by means of a numerical simulation study, and apply a generalized STAR model to a multivariate time series of monthly tea production in west Java, Indonesia.     

On the performance of the ordinary least squares method under an error component model

Starting from the one-dimensional results by Wang et al (1994) we consider the performance of the ordinary least squares estimator in comparison to the best linear unbiased estimator under an error component model with random effects in units and time. Upper bounds are derived for the first-order approximation to the difference between both estimators and for the spectral norm of the difference between their dispersion matrices.     

A note on least squares estimation and the blue in a generalized linear regression model

In a generalized linear regression model, least squares and Gauss-Markov estimators differ, in general, if the variance-covariance matrix of the disturbances is singular. In the present note it is shown that, nevertheless, the conventional least squares procedure leads to a Gauss-Markov estimator if it is applied to a modified model which results from adding dummy constraints to the original model. These constraints reflect the effects of the singularity of the variance- convariance matrix. As a consequence, a Gauss-Markov estimate may always be obtained by standard least squares minimization, which offers considerable computational advantages.     

Efficiency of iterated WLS in the linear model with completely unknown heteroskedasticity

Iterated weighted least squares (IWLS) is investigated for estimating the regression coefficients in a linear model with symmetrically distributed errors. The variances of the errors are not specified; it is not assumed that they are unknown functions of the explanatory variables nor that they are given in some parametric way.
IWLS is carried out in a random number of steps, of which the first one is OLS. In each step the error variance at time t is estimated with a weighted sum of m squared residuals in the neighbourhood of t and the coefficients are estimated using WLS. Furthermore an estimate of the co-variance matrix is obtained. If this estimate is minimal in some way the iteration process is stopped.
Asymptotic properties of IWLS are derived for increasing sample size n . Some particular cases show that the asymptotic efficiency can be increased by allowing more than two steps. Even asymptotic efficiency with respect to WLS with the true error variances can be obtained if m is not fixed but tends to infinity with n and if the heteroskedasticity is smooth.     

A consistent estimator for nonlinear regression models

In this paper an estimator for the general (nonlinear) regression model with random regressors is studied which is based on the Fourier transform of a certain weight function. Consistency and asymptotic normality of the estimator are established and simulation results are presented to illustrate the theoretical ones.Supported by the Hungarian National Science Foundation OTKA under Grants No. F 032060/2000 and F 046061/2004 and by the Bolyai Grant of the Hungarian Academy of Sciences.Received October 2003     

Finite sample properties for the semiparametric estimation of the intercept of a censored regression model

Financial support for this paper was provided by a C.A. Anderson Fellowship of the Cowles Foundation. I wish to thank Donald Andrews, Moshe Buchinsky, Oliver Linton, and Peter Robinson for helpful discussions. I also wish to thank three anonymous referees for their comments and suggestions. I am, of course, responsible for any remaining errors. A popular two-step estimator of the intercept of a censored regression model is compared with consistent asymptotically normal semiparametric alternatives. Using a root mean squared error criterion, the semiparametric estimators perform better for a range of bandwidth parameter choices for a variety of distributions of the errors and regressors. For error distributions that are close to the normal, however, the two-step parametric estimator performs better.     

Consistent estimation in the bilinear multivariate errors-in-variables model

A bilinear multivariate errors-in-variables model is considered. It corresponds to an overdetermined set of linear equations AXB=C, A∈ℝ^m×n, B∈ℝ^p×q, in which the data A, B, C are perturbed by errors. The total least squares estimator is inconsistent in this case.  An adjusted least squares estimator is constructed, which converges to the true value X, as m →∞, q →∞. A small sample modification of the estimator is presented, which is more stable for small m and q and is asymptotically equivalent to the adjusted least squares estimator. The theoretical results are confirmed by a simulation study. Acknowledgements. We thank two anonymous reviewers for their suggestions and corrections.? A. Kukush is supported by a postdoctoral research fellowship of the Belgian office for Scientific, Technical and Cultural Affairs, promoting Scientific and Technical Collaboration with Central and Eastern Europe.? S. Van Huffel is a full professor with the Katholieke Universiteit Leuven.? I. Markovsky is a research assistant with the Katholieke Universiteit Leuven.? This paper presents research results of the Belgian Programme on Interuniversity Poles of Attraction (IUAP V-22), initiated by the Belgian State, Prime Minister's Office – Federal Office for Scientific, Technical and Cultural Affairs of the Concerted Research Action (GOA) projects of the Flemish Government MEFISTO-666 (Mathematical Engineering for Information and Communication Systems Technology), of the IDO/99/03 project (K.U. Leuven) “Predictive computer models for medical classification problems using patient data and expert knowledge”, of the FWO projects G.0078.01, G.0200.00, and G0.0270.02.? The scientific responsibility is assumed by its authors.     

Least squares estimator for regression models with some deterministic time varying parameters

Here we study the least squares estimates in some regression models. We assume that the evolution of the parameter is linearly explosive (i.e. polynomial), or stable (i.e. sinusoidal). We prove the strong consistency, and establish the rate of convergence.     

Testing a linear regression model against nonparametric alternatives

Summary As a test statistic for testing goodness-of-fit of a linear regression model, we propose a ratio of quadratic forms measuring the distance between parametric and nonparametric fits, relative to the estimated error variance. The test statistic is a modification of the statistic suggested by H?rdle and Mammen (1988). The asymptotic distribution under the hypothesis is established. The finite sample behaviour of the test is investigated in a Monte Carlo study, and is illustrated for two applications.     

Concentration behavior of the penalized least squares estimator

Consider the standard nonparametric regression model and take as estimator the penalized least squares function. In this article, we study the trade‐off between closeness to the true function and complexity penalization of the estimator, where complexity is described by a seminorm on a class of functions. First, we present an exponential concentration inequality revealing the concentration behavior of the trade‐off of the penalized least squares estimator around a nonrandom quantity, where such quantity depends on the problem under consideration. Then, under some conditions and for the proper choice of the tuning parameter, we obtain bounds for this nonrandom quantity. We illustrate our results with some examples that include the smoothing splines estimator.     

MSE-improvement of the least squares estimator by dropping variables

It is well known that dropping variables in regression analysis decreases the variance of the least squares (LS) estimator of the remaining parameters. However, after elimination estimates of these parameters are biased, if the full model is correct. In his recent paper, Boscher (1991) showed that the LS-estimator in the special case of a mean shift model (cf. Cook and Weisberg, 1982) which assumes no “outliers” can be considered in the framework of a linear regression model where some variables are deleted. He derived conditions under which this estimator outperforms the LS-estimator of the full model in terms of the mean squared error (MSE)-matrix criterion. We demonstrate that this approach can be extended to the general set-up of dropping variables. Necessary and sufficient conditions for the MSE-matrix superiority of the LS-estimator in the reduced model over that in the full model are derived. We also provide a uniformly most powerful F-statistic for testing the MSE-improvement.     

Variance estimation in the change analysis of a linear regression model

We study a “direct test” of Chu and White (1992) proposed for detecting changes in the trend of a linear regression model. The power of this test strongly depends on a suitable estimation of the variance of the error variables involved. We discuss various types of variance estimators and derive their asymptotic properties under the null-hypothesis of “no change” as well as under the alternative of “a change in linear trend”. A small simulation study illustrates the estimators' finite sample behaviour.     

Partial least squares structural equation modeling in HRM research

Abstract

Partial least squares structural equation modeling (PLS-SEM) has become a key multivariate analysis technique that human resource management (HRM) researchers frequently use. While most disciplines undertake regular critical reflections on the use of important methods to ensure rigorous research and publication practices, the use of PLS-SEM in HRM has not been analyzed so far. To address this gap in HRM literature, this paper presents a critical review of PLS-SEM use in 77 HRM studies published over a 30-year period in leading journals. By contrasting the review results with state-of-the-art guidelines for use of the method, we identify several areas that offer room of improvement when applying PLS-SEM in HRM studies. Our findings offer important guidance for future use of the PLS-SEM method in HRM and related fields.     

Optimality of least squares in the seemingly unrelated regression equation model

In this article the authors have investigated the situations in which the single-equation least squares estimator is identical with the generalized least squares estimator in the seemingly unrelated regression model. The condition obtained turned out to be advantageous from an empirical point of view as it permits one to decide whether to go for a single-equation least squares method or Zellner's method with estimated disturbance variance covariance matrix for estimating the coefficients in the model.     

The analysis of proportions in agricultural experiments by a generalized linear mixed model

This paper is concerned with the statistical analysis of proportions involving extra-binomial variation. Extra-binomial variation is inherent to experimental situations where experimental units are subject to some source of variation, e.g. biological or environmental variation. A generalized linear model for proportions does not account for random variation between experimental units. In this paper an extended version of the generalized linear model is discussed with special reference to experiments in agricultural research. In this model it is assumed that both treatment effects and random contributions of plots are part of the linear predictor. The methods are applied to results from two agricultural experiments.     

Semiparametric time series regression modeling with a diverging number of parameters

Variable selection and error structure determination of a partially linear model with time series errors are important issues. In this paper, we investigate the regression coefficient and autoregressive order shrinkage and selection via the smoothly clipped absolute deviation penalty for a partially linear model with a divergent number of covariates and finite order autoregressive time series errors. Both consistency and asymptotic normality of the proposed penalized estimators are derived. The oracle property of the resultant estimators is proved. Simulation studies are carried out to assess the finite‐sample performance of the proposed procedure. A real data analysis is made to illustrate the usefulness of the proposed procedure as well.     

The effect of a Durbin–Watson pretest on confidence intervals in regression

Consider a linear regression model and suppose that our aim is to find a confidence interval for a specified linear combination of the regression parameters. In practice, it is common to perform a Durbin–Watson pretest of the null hypothesis of zero first‐order autocorrelation of the random errors against the alternative hypothesis of positive first‐order autocorrelation. If this null hypothesis is accepted then the confidence interval centered on the ordinary least squares estimator is used; otherwise the confidence interval centered on the feasible generalized least squares estimator is used. For any given design matrix and parameter of interest, we compare the confidence interval resulting from this two‐stage procedure and the confidence interval that is always centered on the feasible generalized least squares estimator, as follows. First, we compare the coverage probability functions of these confidence intervals. Second, we compute the scaled expected length of the confidence interval resulting from the two‐stage procedure, where the scaling is with respect to the expected length of the confidence interval centered on the feasible generalized least squares estimator, with the same minimum coverage probability. These comparisons are used to choose the better confidence interval, prior to any examination of the observed response vector.     

An exchange algorithm for computing the least quartile difference estimator

We propose an exchange algorithm (EA) for computing the least quartile difference estimate in a multiple linear regression model. Empirical results suggest that the EA is faster and more accurate than the usual p-subset algorithm.