Improved Stein-rule estimator for regression problems

This note extends the asymptotic expansion of the risk of the double k-class estimator of Ullah and Ullah (1978, 1981) and discusses the k₁ and k₂ values which minimize it. An error in Vinod (1980) is also corrected.     

Estimation of regression coefficients after a preliminary test for homoscedasticity

When heteroscedasticity of the variances of disturbances in a regression model is suspected, we perform a preliminary test for homoscedasticity prior to estimation of regression coefficients. According to the result of the pre-test, we use either the ordinary least squares estimator or the two-stage Aitken estimator (2SAE). In this paper, using orthonormal regressors, we derive the mean square error (MSE) of the pre-test estimator and show that the 2SAE is inadmissible when the MSE is used as a criterion. Further, we seek the optimal critical value of the pre-test in the sense of minimizing the average relative risk which is based on the MSE.     

Standardization of Variables and Collinearity Diagnostic in Ridge Regression

Ridge estimation (RE) is an alternative method to ordinary least squares when there exists a collinearity problem in a linear regression model. The variance inflator factor (VIF) is applied to test if the problem exists in the original model and is also necessary after applying the ridge estimate to check if the chosen value for parameter k has mitigated the collinearity problem. This paper shows that the application of the original data when working with the ridge estimate leads to non‐monotone VIF values. García et al. (2014) showed some problems with the traditional VIF used in RE. We propose an augmented VIF, VIF_R(j,k), associated with RE, which is obtained by standardizing the data before augmenting the model. The VIF_R(j,k) will coincide with the VIF associated with the ordinary least squares estimator when k = 0. The augmented VIF has the very desirable properties of being continuous, monotone in the ridge parameter and higher than one.     

Berry-Esséen rate in asymptotic normality for perturbed sample quantiles

In estimating quantiles with a sample of sizeN obtained from a distributionF, the perturbed sample quantiles based on a kernel functionk have been investigated by many authors. It is well known that their behaviour depends on the choices of “window-width”, sayw _N. Under suitable and reasonably mild assumptions onF andk, Ralescu and Sun (1993) have recently proven that lim _N→∞ N ^1/4w_N=0 is the necessary and sufficient condition for the asymptotic normality of the perturbed sample quantiles. In this paper, their rate of convergence is investigated. It turns out that the optimal Berry-Esséen rate ofO(N^?1/2) can be achieved by choosing the window-width suitably, sayw _N=O(N^?1/2). The obtained results, in addition to being explicit enough to verify the sufficient condition for the asymptotic normality, improve Ralescu's (1992) result of which the rate is of order (logN)N ^?1/2.     

Design of optimal step�Cstress accelerated life tests under progressive type I censoring with random removals

This paper considers the analysis of exponential distributed lifetime data observed under k-stage step?Cstress accelerated life test under progressive type I censoring with random removal (PCRR), where the number of units removed at each stress follows a binomial distribution. We compute the expected Fisher information matrix of the maximum likelihood estimator of the log mean life at design (use) stress. The problem of choosing the optimal time ?? _i , i = 1, . . . , k ? 1 under k-stage step?Cstress is addressed using variance (V)-optimality as well as determinant (D)-optimality criteria. An illustrative example is provided with discussion.     

A new interpretation of optimality forE-optimal designs in linear regression models

The optimal design problem for the estimation of several linear combinationsc′ _l ϑ (l=1, …,m) is considered in the usual linear regression modely=f′(x)ϑ (f(x) ∈ ℝ ^k ,ϑ ∈ ℝ ^k ). An optimal design minimizes a (weighted)p-norm of the variances of the least squares estimates for the different linear combinationsc′ _l ϑ. A generalized Elfving theorem is used to derive the relation of the new optimality criterion to theE-optimal design problem. It is shown that theE-optimal design for the parameterϑ minimizes such a (weighted)p-norm whenever the vectorc=(c′ ₁, …, c′_k)′ is an inball vector of a symmetric convex and compact “Elfving set” in.     

Estimation in the first-order moving average model through the finite autoregressive approximation: Some asymptotic results

To estimate α in the model y_t = u_t+αu_t?1, we consider a proposal by Durbin (Biometrika, 1969). It consists in fitting an autoregression of order k to the data, and deriving from there an estimate α^. The probability limit and the variance of the limiting normal distribution of α^ are presented and discussed in detail, when the sample size T → ∞, but k remains fixed. The differences between the resulting values and those corresponding to the maximum likelihood estimator are exponentially decreasing functions of k. Several modifications of the estimator are discussed and found consistent, but asymptotically inefficient.     

On nonparametric latent trait models

Some nonparametric latent trait models for dichotomous data are considered. We deal with n subjects, each answering to the same set of of k items, each item being scored dichotomously. We are interested in ordering item difficulties α₁,...α_k . In Sec. 2 it is shown that in the considered nonparametric models the ordering is identifiable. Then an order estimator is defined and its quality is described by the probabilities of correct, wrong and deferred decision. Asymptotic behaviour of these probabilities are considered for n→∞ and any k≥2. The hypothesis that the probability of wrong decision diminishes when the model is “more distant” from so called random response model, is proved for n≤3 and verified numerically for n≥3. In Sec. 4 we discuss critically some parameters of nonparametric models known in the literature as “coefficients of scalability”. In particular, for k=2 their connections with the evaluation of positive dependence are considered.     

Combining information for prediction in linear regression

This article deals with the prediction problem in linear regression where the measurements are obtained using k different devices or collected from k different independent sources. For the case of k=2, a Graybill-Deal type combined estimtor for the regression parameters is shown to dominate the individual least squares estimators under the covariance criterion. Two predictors ŷ _c and ŷ _p are proposed. ŷ _c is based on a combined estimator of the regression coefficient vector, and ŷ _p is obtained by combining the individual predictors from different models. Prediction mean square errors of both predictors are derived. It is shown that the predictor ŷ _p is better than the individual predictors for k≥2 and the predictor ŷ _c is better than the individual predictors for k=2. Numerical comparison between ŷ _c and ŷ _p shows that the former is superior to the latter for the case k=2.     

A note on moments of k-class estimators for negative k

The validity of expressions for the exact moments of k-class estimator with 0≦1E;k<1 is established for negative values of k in the interval (–1,0). For other negative values (–∞<k≦1E;–1) the derivation of expressions for moments is outlined.     

The effects of autocorrelation among errors on the consistency property of OLS variance estimator

In a linear regression model the ordinary least squares (OLS) variance estimator (S²) converges in probability to E(S²) even when the errors are autocorrelated. Of interest, however, is the rate of convergence. In this paper we shed some light on this question for the case of a linear trend model. In particular the relation between the rate of convergence and the correlation property of the errors is explored. It is shown that the retardation of the rate of convergence is not appreciable if the correlation is moderate, but it can be severe for extreme correlations.     

Asymptotic distributions of some scale estimators in nonlinear models

Often in the robust analysis of regression and time series models there is a need for having a robust scale estimator of a scale parameter of the errors. One often used scale estimator is the median of the absolute residuals s ₁. It is of interest to know its limiting distribution and the consistency rate. Its limiting distribution generally depends on the estimator of the regression and/or autoregressive parameter vector unless the errors are symmetrically distributed around zero. To overcome this difficulty it is then natural to use the median of the absolute differences of pairwise residuals, s ₂, as a scale estimator. This paper derives the asymptotic distributions of these two estimators for a large class of nonlinear regression and autoregressive models when the errors are independent and identically distributed. It is found that the asymptotic distribution of a suitably standardizes s ₂ is free of the initial estimator of the regression/autoregressive parameters. A similar conclusion also holds for s ₁ in linear regression models through the origin and with centered designs, and in linear autoregressive models with zero mean errors.  This paper also investigates the limiting distributions of these estimators in nonlinear regression models with long memory moving average errors. An interesting finding is that if the errors are symmetric around zero, then not only is the limiting distribution of a suitably standardized s ₁ free of the regression estimator, but it is degenerate at zero. On the other hand a similarly standardized s ₂ converges in distribution to a normal distribution, regardless of the errors being symmetric or not. One clear conclusion is that under the symmetry of the long memory moving average errors, the rate of consistency for s ₁ is faster than that of s ₂.     

Uncorrelated residuals from linear models

In the usual linear model $\text{y}\text{=}\text{Xβ}\text{+}\text{u}$, the error vector u is not observable and the vector r of least squares residuals has a singular covariance matrix that depends on the design matrix X. We approximate u by a vector$\text{r}{}^1\text{=}\text{G}\text{(}\text{JA'y}\text{+}\text{Kz}\text{)}$ of uncorrelated ‘residuals’, where G and $\text{(}\text{J, K}\text{)}$ are orthogonal matrices, $\text{A'X}\text{=}\text{0}$ and $\text{A'A}\text{=}\text{I}$, while z is either 0 or a random vector uncorrelated with u satisfying $\text{E(}\text{z}\text{) = E(}\text{J'u}\text{) =}\text{0}$, $\text{V(}\text{z}\text{) = V(}\text{J'u}\text{) = σ}{}^2\text{I}$. We prove that $\text{r}{}^1\text{-}\text{r}$ is uncorrelated with $\text{r}\text{-}\text{u}$, for any such r¹, extending the results of Neudecker (1969). Building on results of Hildreth (1971) and Tiao and Guttman (1967), we show that the BAUS residual vector $\text{r}{}_{\mathrm{h}}\text{=}\text{r}\text{+}\text{P}{}_{\mathrm{<mtext>1</mtext>}}\text{z}$, where P₁ is an orthonormal basis for X, minimizes each characteristic root of $\text{V(}\text{r}{}^1\text{-}\text{u}\text{)}$, while the vector r_b of Theil's BLUS residuals minimizes each characteristic root of $\text{V(}\text{Jr}{}_{\mathrm{a}}\text{-}\text{r}\text{)}$, cf. Grossman and Styan (1972). We find that $\text{tr V(}\text{r}{}_{\mathrm{h}}\text{-}\text{u}\text{) < tr V(}\text{Jr}{}_{\mathrm{b}}\text{-}\text{u}\text{)}$ if and only if the average of the singular values of $\text{P}\text{′}{}_1\text{K}$ is less than $\text{1}\text{2}$, and give examples to show that BAUS is often better than BLUS in this sense.     

A note on reliability properties of k-record statistics

LetY _k,n denote the nth (upper) k-record value of an infinite sequence of independent, identically distributed random variables with common continuous distribution function F. We show that if the nth k-record valueY _k,n has an increasing failure rate (IFR), thenY _l,n (l<k) andY _k+1,n+1(n≤k+1) also have IFR distributions. On the other hand, ifY _k,n has a decreasing failure rate (DFR), thenY _l,n (1>k) has also a DFR distribution. We also present some results concerning log convexity and log concavity ofY _k,n .     

The graph of the k-class estimator: An algebraic and statistical interpretation

This paper presents an algebraic analysis of the graphs of the k-class estimator, its asymptotic standard error and asymptotic t-ratio as functions of k for a single structural equation containing one or more endogenous explanatory variables. These results are illustrated by the corresponding graphs of the second and fifth equations of the Girshick-Haavelmo (1947) Demand for Food Model.Tests of the rank condition for identification are also developed. They are found to involve the values of k which explode the k-class estimator.     

On bootstrapping the mode in the nonparametric regression model with random design

In the nonparametric regression model with random design and based on i.i.d. pairs of observations (X _i, Y _i), where the regression function m is given by m(x)=?(Y _i|X _i=x), estimation of the location θ (mode) of a unique maximum of m by the location of a maximum of the Nadaraya-Watson kernel estimator for the curve m is considered. In order to obtain asymptotic confidence intervals for θ, the suitably normalized distribution of is bootstrapped in two ways: we present a paired bootstrap (PB) where resampling is done from the empirical distribution of the pairs of observations and a smoothed paired bootstrap (SPB) where the bootstrap variables are generated from a smooth bivariate density based on the pairs of observations. While the PB requires only relatively small computational effort when carried out in practice, it is shown to work only in the case of vanishing asymptotic bias, i.e. of “undersmoothing” when compared to optimal smoothing for mode estimation. On the other hand, the SPB, although causing more intricate computations, is able to capture the correct amount of bias if the pilot estimator for m oversmoothes. Received: May 2000     

Utility functions for Debreu's ‘excess demands’

Given an arbitrary function x: $\text{R}$^l→$\text{R}$^l satisfying Walras law and homogeneity, Debreu decomposed x into the sum of l ‘individually rational’ functions x(p)=Σ^l_k=1[uvbar|x]^k(p). Here we find explicit utility functions u^k, constructed on the basis of a simple geometric intuition, which give rise to Debreu's excess demands [uvbar|x]^k(p).     

Strong uniform convergence for the estimator of the regression function under ϕ-mixing conditions

Suppose the observations (X _i,Y _i), i=1,…, n, are ϕ-mixing. The strong uniform convergence and convergence rate for the estimator of the regression function was studied by serveral authors, e.g. G. Collomb (1984), L. Gy?rfi et al. (1989). But the optimal convergence rates are not reached unless the Y _i are bounded or the E exp (a|Y _i|) are bounded for some a>0. Compared with the i.i.d. case the convergence of the Nadaraya-Watson estimator under ϕ-mixing variables needs strong moment conditions. In this paper we study the strong uniform convergence and convergence rate for the improved kernel estimator of the regression function which has been suggested by Cheng P. (1983). Compared with Theorem A in Y. P. Mack and B. Silverman (1982) or Theorem 3.3.1 in L. Gy?rfi et al. (1989), we prove the convergence for this kind of estimators under weaker moment conditions. The optimal convergence rate for the improved kernel estimator is attained under almost the same conditions of Theorem 3.3.2 in L. Gy?rfi et al. (1989). Received: September 1999     

Revelations of a gambler

Knowing that a decision maker maximizes expected utility with respect to some (unknown) utility U and some (unknown) probability P, what can one tell about P by observing his decisions? We discuss this revealed preference question primarily in the simple case of a two-element (H and T) state space, and show that the possible revelations of P_T/P_H are precisely those of the form P_T/P_Hε∪^K_k=1 (γ_k,δ_k), for some algebraic numbers γ_k,δ_k.