On kalman filtering,posterior mode estimation and fisher scoring in dynamic exponential family regression

Summary Dynamic exponential family regression provides a framework for nonlinear regression analysis with time dependent parametersβ ₀,β ₁, …,β _t, …, dimβ _t=p. In addition to the familiar conditionally Gaussian model, it covers e.g. models for categorical or counted responses. Parameters can be estimated by extended Kalman filtering and smoothing. In this paper, further algorithms are presented. They are derived from posterior mode estimation of the whole parameter vector (β′₀, …,β′_t) by Gauss-Newton resp. Fisher scoring iterations. Factorizing the information matrix into block-bidiagonal matrices, algorithms can be given in a forward-backward recursive form where only inverses of “small”p×p-matrices occur. Approximate error covariance matrices are obtained by an inversion formula for the information matrix, which is explicit up top×p-matrices. Heinz Leo Kaufmann, my friend and coauthor for many years, died in a tragical rock climbing accident in August 1989. This paper is dedicated to his memory.     

On bootstrapping the mode in the nonparametric regression model with random design

In the nonparametric regression model with random design and based on i.i.d. pairs of observations (X _i, Y _i), where the regression function m is given by m(x)=?(Y _i|X _i=x), estimation of the location θ (mode) of a unique maximum of m by the location of a maximum of the Nadaraya-Watson kernel estimator for the curve m is considered. In order to obtain asymptotic confidence intervals for θ, the suitably normalized distribution of is bootstrapped in two ways: we present a paired bootstrap (PB) where resampling is done from the empirical distribution of the pairs of observations and a smoothed paired bootstrap (SPB) where the bootstrap variables are generated from a smooth bivariate density based on the pairs of observations. While the PB requires only relatively small computational effort when carried out in practice, it is shown to work only in the case of vanishing asymptotic bias, i.e. of “undersmoothing” when compared to optimal smoothing for mode estimation. On the other hand, the SPB, although causing more intricate computations, is able to capture the correct amount of bias if the pilot estimator for m oversmoothes. Received: May 2000     

Asymptotic results and tests for the choice of approximative models in nonlinear two-phases regression models, heteroscedatic case

This paper is devoted to the study of the least squares estimator of f for the classical, fixed design, nonlinear model X (t _i)=f(t _i)+ε(t _i), i=1,2,…,n, where the (ε(t _i))_i=1,…,n are independent second order r.v.. The estimation of f is based upon a given parametric form. In Brodeau (1993) this subject has been studied in the homoscedastic case. This time we assume that the ε(t _i) have non constant and unknown variances σ²(t _i). Our main goal is to develop two statistical tests, one for testing that f belongs to a given class of functions possibly discontinuous in their first derivative, and another for comparing two such classes. The fundamental tool is an approximation of the elements of these classes by more regular functions, which leads to asymptotic properties of estimators based on the least squares estimator of the unknown parameters. We point out that Neubauer and Zwanzig (1995) have obtained interesting results for connected subjects by using the same technique of approximation. Received: February 1996     

On the power function of the exact test for ther×c contingency table

Summary LetN=[n _ij ] (i=1, …,r;j=1, …,c) be the matrix of observed frequencies in anr×c contingency table fromr possibly different multinomial populations with respective probabilitiesp _i =(p _i1, …,p _ic ).Freeman andHalton have proposed an exact conditional test for the hypothesisH ₀ :p _i =(p ₁, …p _c ) of the exact test is derived. Numerical values forβ(p) were previously computed for the special case:r=3,c=2 [Bennett andNakamura, 1964].     

Combining information for prediction in linear regression

This article deals with the prediction problem in linear regression where the measurements are obtained using k different devices or collected from k different independent sources. For the case of k=2, a Graybill-Deal type combined estimtor for the regression parameters is shown to dominate the individual least squares estimators under the covariance criterion. Two predictors ŷ _c and ŷ _p are proposed. ŷ _c is based on a combined estimator of the regression coefficient vector, and ŷ _p is obtained by combining the individual predictors from different models. Prediction mean square errors of both predictors are derived. It is shown that the predictor ŷ _p is better than the individual predictors for k≥2 and the predictor ŷ _c is better than the individual predictors for k=2. Numerical comparison between ŷ _c and ŷ _p shows that the former is superior to the latter for the case k=2.     

Optimal design of nonlinear experiments with parameter constraints

The nonlinear regression model with N observations y _i=η(x _i,θ) +ε_i, and with the parameter θ subject to q nonlinear constraints C _j (θ)=0; j=1, …,q, is considered. As an example, the spline regression with unknown nodes is taken. Expressions for the variances (variance matrices) of the LSE are discussed. Because of the complexity of these expressions, and the singularity of the variance matrix of the LSE for θ, the optimality criteria and their properties, in particular the convexity and the equivalence theorem are considered from different aspects. Also the possibility of restriction to designs with limited values of measures of nonlinearity is mentioned. Research supported by the VEGA-grant of the Slovak grant agency No. 1/7295/20.     

Asymptotic normality of M-estimators in a semiparametric model with longitudinal data

Suppose that the longitudinal observations (Y _ij , X _ij , t _ij ) for i = 1, . . . ,n; j = 1, . . . ,m _i are modeled by the semiparamtric model where β ₀ is a k × 1 vector of unknown parameters, g(·) is an unknown estimated function and e _ij are unobserved disturbances. This article consider M-type regressions which include mean, median and quantile regressions. The M-estimator of the slope parameter β ₀ is obtained through piecewise local polynomial approximation of the nonparametric component. The local M-estimator of g(·) is also obtained by replacing β ₀ in model with its M-estimator and using local linear approximation. The asymptotic distribution of the estimator of β ₀ is derived. The asymptotic distributions of the local M-estimators of g(·) at both interior and boundary points are also established. Various applications of our main results are given. The research is supported in part by National Natural Science Foundation of China (Grant No. 10671089).     

Strong uniform convergence for the estimator of the regression function under ϕ-mixing conditions

Suppose the observations (X _i,Y _i), i=1,…, n, are ϕ-mixing. The strong uniform convergence and convergence rate for the estimator of the regression function was studied by serveral authors, e.g. G. Collomb (1984), L. Gy?rfi et al. (1989). But the optimal convergence rates are not reached unless the Y _i are bounded or the E exp (a|Y _i|) are bounded for some a>0. Compared with the i.i.d. case the convergence of the Nadaraya-Watson estimator under ϕ-mixing variables needs strong moment conditions. In this paper we study the strong uniform convergence and convergence rate for the improved kernel estimator of the regression function which has been suggested by Cheng P. (1983). Compared with Theorem A in Y. P. Mack and B. Silverman (1982) or Theorem 3.3.1 in L. Gy?rfi et al. (1989), we prove the convergence for this kind of estimators under weaker moment conditions. The optimal convergence rate for the improved kernel estimator is attained under almost the same conditions of Theorem 3.3.2 in L. Gy?rfi et al. (1989). Received: September 1999     

Moment consistency of estimators in partially linear models under NA samples

Consider the heteroscedastic regression model Y ^(j)(x _in , t _in ) = t _in β + g(x _in ) + σ _in e ^(j)(x _in ), 1 ≤ j ≤ m, 1 ≤ i ≤ n, where s_in²=f(u_in){\sigma_{in}^{2}=f(u_{in})}, (x _in , t _in , u _in ) are fixed design points, β is an unknown parameter, g(·) and f(·) are unknown functions, and the errors {e ^(j)(x _in )} are mean zero NA random variables. The moment consistency for least-squares estimators and weighted least-squares estimators of β is studied. In addition, the moment consistency for estimators of g(·) and f(·) is investigated.     

Largest excess of boundary crossings for martingales

Summary LetX ₁,X ₂, ... form a sequence of martingale differences and denote byZ(a, α) = sup n (S _n −an ^α)⁺ the largest excess forS _n =X ₁ + ... +X _n crossing the boundaryan ^α. We give a sufficient condition for the finiteness ofEZ(a, α)^β which is formulated in terms of bounds forE(X _i ^{+ p} andE(|X _i |γ|X ₁, ...,X _i-1), whereα, β, γ, p are suitably related. This general result is then applied to the case of independent random variables.     

Strong consistency of the MLE under random censoring

LetX ₁, ...,X _n be an i.i.d. sample from some parametric family {_θ :θ (Θ} of densities. In the random censorship model one observesZ _i =min (X _i ,Y _i ) andδ _i =1_{ x _i ≤Y _i}, whereY _i is a censoring variable being independent ofX _i . In this paper we investigate the strong consistency ofθ _n maximizing the modified likelihood function based on (Z _i ,δ _i , 1≤i≤n. The main result constitutes an extension of Wald’s theorem for complete data to censored data. Work partially supported by the “Deutsche Forschungsgemeinschaft”.     

Goodness-of-fit test with nuisance regression and scale

In the linear model Y _i = x _i + e _i, i=1,,n, with unknown (, ), {\open R}^p, >0, and with i.i.d. errors e ₁,,e _n having a continuous distribution F, we test for the goodness-of-fit hypothesis H ₀:F(e)F ₀(e/), for a specified symmetric distribution F ₀, not necessarily normal. Even the finite sample null distribution of the proposed test criterion is independent of unknown (,), and the asymptotic null distribution is normal, as well as the distribution under local (contiguous) alternatives. The proposed tests are consistent against a general class of (nonparametric) alternatives, including the case of F having heavier (or lighter) tails than F ₀. A simulation study illustrates a good performance of the tests. Received July 2001     

A new interpretation of optimality forE-optimal designs in linear regression models

The optimal design problem for the estimation of several linear combinationsc′ _l ϑ (l=1, …,m) is considered in the usual linear regression modely=f′(x)ϑ (f(x) ∈ ℝ ^k ,ϑ ∈ ℝ ^k ). An optimal design minimizes a (weighted)p-norm of the variances of the least squares estimates for the different linear combinationsc′ _l ϑ. A generalized Elfving theorem is used to derive the relation of the new optimality criterion to theE-optimal design problem. It is shown that theE-optimal design for the parameterϑ minimizes such a (weighted)p-norm whenever the vectorc=(c′ ₁, …, c′_k)′ is an inball vector of a symmetric convex and compact “Elfving set” in.     

Multivariate likelihood ratio orderings between spacings of heterogeneous exponential random variables

Let X ₁, X ₂, ..., X _n be independent exponential random variables such that X _i has failure rate λ for i = 1, ..., p and X _j has failure rate λ^* for j = p + 1, ..., n, where p ≥ 1 and q = n − p ≥ 1. Denote by D _i:n (p,q) = X _i:n −X _i-1:n the ith spacing of the order statistics X _1:n ≤ X _2:n ≤ ... ≤ X _n:n , i = 1, ..., n, where X _0:n ≡ 0. The purpose of this paper is to investigate multivariate likelihood ratio orderings between spacings D _i:n (p,q), generalizing univariate comparison results in Wen et al.(J Multivariate Anal 98:743–756, 2007). We also point out that such multivariate likelihood ratio orderings do not hold for order statistics instead of spacings. Supported by National Natural Science Foundation of China, the Program for New Century Excellent Talents in University (No.: NCET-04-0569), and by the Knowledge Innovation Program of the Chinese Academy of Sciences (No.: KJCX3-SYW-S02).     

Estimation of a regression coefficient after two preliminary tests of significance

Summary The estimation of the regression coefficient of a population, defined byE (y)= +x, incorporating two preliminary tests of significance has been discussed. The experimenter has two random samples of different sizes from two such populations, as defined above, with regression coefficients ₁ and ₂ respectively, where ₂ may possibly be equal to ₁. Besides this, it is also conjectured that the common conditional variance ² of the two populations has a specified value ₀ ² . The two preliminary tests are used to resolve these two uncertainties.The author has rejoined Lucknow University, Lucknow India on Oct. 4, 1976 after availing leave for two years.     

Locally minimax test of independence in elliptically symmetrical distributions with additional observations

Summary LetX=(X _ij )=(X ₁, ...,X _n )’,X’ _i =(X _i1, ...,X _ip )’,i=1,2, ...,n be a matrix having a multivariate elliptical distribution depending on a convex functionq with parameters, 0,σ. Let ϱ²=ϱ ₂ ^-2 be the squared multiple correlation coefficient between the first and the remainingp ₂+p ₃=p−1 components of eachX _i . We have considered here the problem of testingH ₀:ϱ²=0 against the alternativesH ₁:ϱ ₁ ^-2 =0, ϱ ₂ ^-2 >0 on the basis ofX andn ₁ additional observationsY ₁ (n ₁×1) on the first component,n ₂ observationsY ₂(n ₂×p ₂) on the followingp ₂ components andn ₃ additional observationsY ₃(n ₃×p ₃) on the lastp ₃ components and we have derived here the locally minimax test ofH ₀ againstH ₁ when ϱ ₂ ^-2 →0 for a givenq. This test, in general, depends on the choice ofq of the familyQ of elliptically symmetrical distributions and it is not optimality robust forQ.     

Minimal sufficient statistics for the group divisible partially balanced incomplete block design (GD-PBIBD) with interaction under an Eisenhart Model II

Summary In this paper, an Eisenmhart Model II with interaction for a GD-PBIB design withp replicates per cell is considered. Specifically the Model Y_ijl=µ+_i+_j+()_ij+e_ijl is assumed, wherei=1, 2, ...,b; j=1, 2, ...,t andl=0, 1, 2, ...p s _ij wheres _ij=1, if treatmentj appears in blocki, 0, otherwise.If _i, _j, ()_ij ande _ijl are normally and independently distributed, then a minimal sufficient (Vector-valued) statistic for the class of densities for this model is found, together with the distribution of each component in the minimal sufficient statistic. It is also shown that the minimal sufficient statistic for this class densities is not complete. Hence the solution of the problem of finding minimum variance unbiased estimators of the variance components is not straightforward.     

Games of chance with multiple objectives

We consider games of chance between two players: Player M can win only by amassing point totals in several categories before player N scores a prescribed total number n of points. Let M have k objectives, with m _i points required in category i and probability q _i of scoring a point in that category. We resolve certain special cases: (a) For all m _i equal, the probabilities of M winning are ordered by majorization of the vectors (q ₁,...,q _k ). (b) For all q _i equal, the probabilities of M winning are ordered by majorization of the vectors (m ₁,...,m _k ). (c) For all m _i equal and all q _i equal, the probability of M winning approaches 0 as n → ∞ or as k → ∞. The results, which follow from inequalities of majorization and Schur convexity, are in accord with intuition.      

Collapsibility hypotheses and diagnostic bounds in regression analysis

Deleted-case diagnostic statistics in regression analysis are based on changes in estimates due to deleting one or more cases. Bounds on these statistics, suggested in the literature for identifying influential cases, are widely used.  In a linear regression model for Y in terms of X and Z, the model is “collapsible” with respect to Z if the Y−X relation is unchanged by deleting Z from the model. Deleted-case diagnostic statistics can be viewed as test statistics for collapsibility hypotheses in the mean shift outlier model. It follows that, for any given case, all deleted-case statistics test the same hypothesis, hence all have the same p-value, while the bounds correspond to different levels of significance among the several statistics. Furthermore, the bound for any particular deleted-case statistic gives widely varying levels of significance over the cases in the data set. Received: April 1999     

An approach to fuzzy hypothesis testing

In this paper an approach is presented how to test fuzzily formulated hypotheses with crisp data. The quantitiesα andβ, the probabilities of the errors of type I and of type II, are suitably generalized and the concept of a best test is introduced. Within the framework of a one-parameter exponential distribution family the search for a best test is considerably reduced. Furthermore, it is shown under very weak conditions thatα andβ can simultaneously be diminished by increasing the sample size even in the case of testingH ₀ against the omnibus alternativeH ₁: notH ₀, a result completely different from the case of crisp setsH ₀ andH ₁: notH ₀.