Strong uniform convergence of the recursive regression estimator under φ-mixing conditions

Suppose the observations (X_i, Y_i) taking values in R^d×R, are -mixing. Compared with the i.i.d. case, some known strong uniform convergence results for the estimators of the regression function r(x)=E(Y_i|X_i=x) need strong moment conditions under -mixing setting. We consider the following improved kernel estimators of r(x) suggested by Cheng (1983): Qian and Mammitzsch (2000) investigated the strong uniform convergence and convergence rate for to r(x) under weaker moment conditions than those of the others in the literature, and the optimal convergence rate can be attained under almost the same conditions as stated in Theorem 3.3.2 of Györfi et al. (1989). In this paper, under the similar conditions of Qian and Mammitzsch (2000), we study the strong uniform convergence and convergence rates for (j=2,3) to r(x), which have not been discussed by Qian and Mammitzsch (2000). In contrast to , our estimators and are recursive, which is highly desirable for practical computation.     

Density estimation in the uniform deconvolution model

We consider the problem of estimating a probability density function based on data that are corrupted by noise from a uniform distribution. The (nonparametric) maximum likelihood estimator for the corresponding distribution function is well defined. For the density function this is not the case. We study two nonparametric estimators for this density. The first is a type of kernel density estimate based on the empirical distribution function of the observable data. The second is a kernel density estimate based on the MLE of the distribution function of the unobservable (uncorrupted) data.     

Strong uniform convergence for the estimator of the regression function under ϕ-mixing conditions

Suppose the observations (X _i,Y _i), i=1,…, n, are ϕ-mixing. The strong uniform convergence and convergence rate for the estimator of the regression function was studied by serveral authors, e.g. G. Collomb (1984), L. Gy?rfi et al. (1989). But the optimal convergence rates are not reached unless the Y _i are bounded or the E exp (a|Y _i|) are bounded for some a>0. Compared with the i.i.d. case the convergence of the Nadaraya-Watson estimator under ϕ-mixing variables needs strong moment conditions. In this paper we study the strong uniform convergence and convergence rate for the improved kernel estimator of the regression function which has been suggested by Cheng P. (1983). Compared with Theorem A in Y. P. Mack and B. Silverman (1982) or Theorem 3.3.1 in L. Gy?rfi et al. (1989), we prove the convergence for this kind of estimators under weaker moment conditions. The optimal convergence rate for the improved kernel estimator is attained under almost the same conditions of Theorem 3.3.2 in L. Gy?rfi et al. (1989). Received: September 1999     

Combining kernel estimators in the uniform deconvolution problem

We construct a density estimator and an estimator of the distribution function in the uniform deconvolution model. The estimators are based on inversion formulas and kernel estimators of the density of the observations and its derivative. Initially the inversions yield two different estimators of the density and two estimators of the distribution function. We construct asymptotically optimal convex combinations of these two estimators. We also derive pointwise asymptotic normality of the resulting estimators, the pointwise asymptotic biases and an expansion of the mean integrated squared error of the density estimator. It turns out that the pointwise limit distribution of the density estimator is the same as the pointwise limit distribution of the density estimator introduced by Groeneboom and Jongbloed (Neerlandica, 57, 2003, 136), a kernel smoothed nonparametric maximum likelihood estimator of the distribution function.     

Relative efficiency and deficiency of kernel type estimators of smooth distribution functions

Abstract  The problem is investigated whether a given kernel type estimator of a distribution function at a single point has asymptotically better performance than the empirical estimator. A representation of the relative deficiency of the empirical distribution function with respect to a kernel type estimator is established which gives a complete solution to this problem. The problem of finding optimal kernels is studied in detail.     

On estimating nonnegative definite quadratic forms

It is often required to estimate a quadratic form in survey sampling, especially when one has to estimate the mean squared error of a linear estimator of the population total. In this note we consider the problem of obtaining uniformly nonnegative quadratic unbiased estimators for nonnegative definite quadratic forms. The estimators considered here are necessarily quadratic. Received January 1997     

Pseudo empirical likelihood method in the presence of missing data

In this paper, we propose an estimator for the population mean when some observations on the study and auxiliary variables are missing from the sample. The proposed estimator is valid for any unequal probability sampling design, and is based upon the pseudo empirical likelihood method. The proposed estimator is compared with other estimators in a simulation study.     

A consistent estimator for nonlinear regression models

In this paper an estimator for the general (nonlinear) regression model with random regressors is studied which is based on the Fourier transform of a certain weight function. Consistency and asymptotic normality of the estimator are established and simulation results are presented to illustrate the theoretical ones.Supported by the Hungarian National Science Foundation OTKA under Grants No. F 032060/2000 and F 046061/2004 and by the Bolyai Grant of the Hungarian Academy of Sciences.Received October 2003     

Strong Consistency of the Maximum Likelihood Estimator in Generalized Linear and Nonlinear Mixed-Effects Models

Generalized linear and nonlinear mixed-effects models are used extensively in biomedical, social, and agricultural sciences. The statistical analysis of these models is based on the asymptotic properties of the maximum likelihood estimator. However, it is usually assumed that the maximum likelihood estimator is consistent, without providing a proof. A rigorous proof of the consistency by verifying conditions from existing results can be very difficult due to the integrated likelihood. In this paper, we present some easily verifiable conditions for the strong consistency of the maximum likelihood estimator in generalized linear and nonlinear mixed-effects models. Based on this result, we prove that the maximum likelihood estimator is consistent for some frequently used models such as mixed-effects logistic regression models and growth curve models.     

Subsampling realised kernels

In a recent paper we have introduced the class of realised kernel estimators of the increments of quadratic variation in the presence of noise. We showed that this estimator is consistent and derived its limit distribution under various assumptions on the kernel weights. In this paper we extend our analysis, looking at the class of subsampled realised kernels and we derive the limit theory for this class of estimators. We find that subsampling is highly advantageous for estimators based on discontinuous kernels, such as the truncated kernel. For kinked kernels, such as the Bartlett kernel, we show that subsampling is impotent, in the sense that subsampling has no effect on the asymptotic distribution. Perhaps surprisingly, for the efficient smooth kernels, such as the Parzen kernel, we show that subsampling is harmful as it increases the asymptotic variance. We also study the performance of subsampled realised kernels in simulations and in empirical work.     

Three Sides of Smoothing: Categorical Data Smoothing, Nonparametric Regression, and Density Estimation

The past forty years have seen a great deal of research into the construction and properties of nonparametric estimates of smooth functions. This research has focused primarily on two sides of the smoothing problem: nonparametric regression and density estimation. Theoretical results for these two situations are similar, and multivariate density estimation was an early justification for the Nadaraya-Watson kernel regression estimator.
A third, less well-explored, strand of applications of smoothing is to the estimation of probabilities in categorical data. In this paper the position of categorical data smoothing as a bridge between nonparametric regression and density estimation is explored. Nonparametric regression provides a paradigm for the construction of effective categorical smoothing estimates, and use of an appropriate likelihood function yields cell probability estimates with many desirable properties. Such estimates can be used to construct regression estimates when one or more of the categorical variables are viewed as response variables. They also lead naturally to the construction of well-behaved density estimates using local or penalized likelihood estimation, which can then be used in a regression context. Several real data sets are used to illustrate these points.     

Estimating a restricted normal mean

Let X ₁, X ₂, ..., X _n be a random sample from a normal distribution with unknown mean μ and known variance σ ². In many practical situations, μ is known a priori to be restricted to a bounded interval, say [−m, m] for some m > 0. The sample mean , then, becomes an inadmissible estimator for μ. It is also not minimax with respect to the squared error loss function. Minimax and other estimators for this problem have been studied by Casella and Strawderman (Ann Stat 9:870–878, 1981), Bickel (Ann Stat 9:1301–1309, 1981) and Gatsonis et al. (Stat Prob Lett 6:21–30, 1987) etc. In this paper, we obtain some new estimators for μ. The case when the variance σ ² is unknown is also studied and various estimators for μ are proposed. Risk performance of all estimators is numerically compared for both the cases when σ ² may be known and unknown.     

Necessary and sufficient conditions for consistency of generalizedM-estimates

GeneralizedM-estimates (minimum contrast estimates) and their asymptotically equivalent approximate versions are considered. A relatively simple condition is found which is equivalent with consistency of all approximateM-estimates under wide assumptions about the model. This condition is applied in several directions. (i) A more easily verifiable condition equivalent with consistency of all approximateM-estimates is derived and illustrated on models with stationary and ergodic observations. (ii) A condition sufficient for inconsistency of all approximateM-estimates is obtained and illustrated on models with i.i.d. observations. (iii) A simple necessary and sufficient condition for consistency of all approximateM-estimates in linear regression with i.i.d. errors is found. This condition is weaker than sufficient conditions for consistency ofM-estimators known from the literature. A linear regression example is presented where theM-estimate is consistent and an approximateM-estimate is incosistent.Supported by CSAS grant N. 17503.