Algorithms for optimal design with application to multiple polynomial regression

The approximate theory of optimal linear regression design leads to specific convex extremum problems for numerical solution. A conceptual algorithm is stated, whose concrete versions lead us from steepest descent type algorithms to improved gradient methods, and finally to second order methods with excellent convergence behaviour. Applications are given to symmetric multiple polynomial models of degree three or less, where invariance structures are utilized. A final section is devoted to the construction of efficientexact designs of sizeN from the optimal approximate designs. For the multifactor cubic model and some of the most popular optimality criteria (D-, A-, andI-criteria) fairly efficient exact designs are obtained, even for small sample sizeN. AMS Subject Classification: 62K05.Abbreviated Title: Algorithms for Optimal Design.Invited paper presented at the International Conference on Mathematical Statistics,ProbaStat '94, Smolenice, Slovakia.     

Local polynomial estimation of nonparametric simultaneous equations models

We define a new procedure for consistent estimation of nonparametric simultaneous equations models under the conditional mean independence restriction of Newey et al. [1999. Nonparametric estimation of triangular simultaneous equation models. Econometrica 67, 565–603]. It is based upon local polynomial regression and marginal integration techniques. We establish the asymptotic distribution of our estimator under weak data dependence conditions. Simulation evidence suggests that our estimator may significantly outperform the estimators of Pinkse [2000. Nonparametric two-step regression estimation when regressors and errors are dependent. Canadian Journal of Statistics 28, 289–300] and Newey and Powell [2003. Instrumental variable estimation of nonparametric models. Econometrica 71, 1565–1578].     

Efficient discriminating design for a class of nested polynomial regression models

This paper studies efficient designs for simultaneous model discrimination among polynomial regression models up to degree k. Based on the -optimality criterion proposed by Dette (Ann Stat 22:890–903, 1994), a maximin -optimal discriminating design is derived in terms of canonical moments for . Theoretical and numerical results show that the proposed design performs well for model discrimination in most of the considered models.     

Delay times of sequential procedures for multiple time series regression models

We consider a multiple regression model in which the explanatory variables are specified by time series. To sequentially test for the stability of the regression parameters in time, we introduce a detector which is based on the first excess time of a CUSUM-type statistic over a suitably constructed threshold function. The aim of this paper is to study the delay time associated with this detector. As our main result, we derive the limit distribution of the delay time and provide thereby a theory that extends the benchmark average run-length concept utilized in most of the sequential monitoring literature. To highlight the applicability of the limit results in finite samples, we present a Monte Carlo simulation study and an application to macroeconomic data.     

Sequential fixed-width confidence bands for distribution functions under random censoring

Summary Sequential fixed-width confidence bands for a distribution function are derived in the case, when the data are censored from the right. The Breslow-Crowley invariance principle for the Kaplan Meier estimate is extended to the random sample size situation. Also some simulation results are reported, which illustrate the behavior of the stopping times.     

Uniform confidence bands for functions estimated nonparametrically with instrumental variables

This paper is concerned with developing uniform confidence bands for functions estimated nonparametrically with instrumental variables. We show that a sieve nonparametric instrumental variables estimator is pointwise asymptotically normally distributed. The asymptotic normality result holds in both mildly and severely ill-posed cases. We present methods to obtain a uniform confidence band and show that the bootstrap can be used to obtain the required critical values. Monte Carlo experiments illustrate the finite-sample performance of the uniform confidence band.     

Selecting local models in multiple regression by maximizing power

This paper considers multiple regression procedures for analyzing the relationship between a response variable and a vector of d covariates in a nonparametric setting where tuning parameters need to be selected. We introduce an approach which handles the dilemma that with high dimensional data the sparsity of data in regions of the sample space makes estimation of nonparametric curves and surfaces virtually impossible. This is accomplished by abandoning the goal of trying to estimate true underlying curves and instead estimating measures of dependence that can determine important relationships between variables. These dependence measures are based on local parametric fits on subsets of the covariate space that vary in both dimension and size within each dimension. The subset which maximizes a signal to noise ratio is chosen, where the signal is a local estimate of a dependence parameter which depends on the subset dimension and size, and the noise is an estimate of the standard error (SE) of the estimated signal. This approach of choosing the window size to maximize a signal to noise ratio lifts the curse of dimensionality because for regions with sparsity of data the SE is very large. It corresponds to asymptotically maximizing the probability of correctly finding nonspurious relationships between covariates and a response or, more precisely, maximizing asymptotic power among a class of asymptotic level αt-tests indexed by subsets of the covariate space. Subsets that achieve this goal are called features. We investigate the properties of specific procedures based on the preceding ideas using asymptotic theory and Monte Carlo simulations and find that within a selected dimension, the volume of the optimally selected subset does not tend to zero as n → ∞ unless the volume of the subset of the covariate space where the response depends on the covariate vector tends to zero.     

Optimal and robust designs for trigonometric regression models

This article presents discussions on the optimal and robust designs for trigonometric regression models under different optimality criteria. First, we investigate the classical Q-optimal designs for estimating the response function in a full trigonometric regression model with a given order. The equivalencies of Q-, A-, and G-optimal designs for trigonometric regression in general are also articulated. Second, we study minimax designs and their implementation in the case of trigonometric approximation under Q-, A-, and D-optimality. Then, We indicate the existence of the symmetric designs that are D-optimal minimax designs for general trigonometric regression models, and prove the existence of the symmetric designs that are Q- or A-optimal minimax designs for two particular trigonometric regression models under certain conditions.     

Identification and estimation of polynomial errors-in-variables models

Methods of estimation of regression coefficients are proposed when the regression function includes a polynomial in a ‘true’ regressor which is measured with error. Two sources of additional information concerning the unobservable regressor are considered: either an additional indicator of the regressor (itself measured with error) or instrumental variables which characterize the systematic variation in the true regressor. In both cases, estimators are constructed by relating moments involving the unobserved variables to moments of observables; these relations lead to recursion formulae for computation of the regression coefficients and nuisance parameters (e.g., moments of the measurement error). Consistency and asymptotic normality of the estimated coefficients is demonstrated, and consistent estimators of the asymptotic covariant matrices are provided.     

A consistent estimator for nonlinear regression models

In this paper an estimator for the general (nonlinear) regression model with random regressors is studied which is based on the Fourier transform of a certain weight function. Consistency and asymptotic normality of the estimator are established and simulation results are presented to illustrate the theoretical ones.Supported by the Hungarian National Science Foundation OTKA under Grants No. F 032060/2000 and F 046061/2004 and by the Bolyai Grant of the Hungarian Academy of Sciences.Received October 2003     

Useful invariance results for generalized regression models

Certain exact finite-sample invariance results are established for the usual estimators and test statistics in the generalized regression model with non-scalar covariance matrix. By inferring properties of an estimator from the criterion function which defines it, results can be obtained even when there is no explicit solution for the estimator as a function of the data (as, e.g., with maximum likelihood). Applications are illustrated by an examination of some recently published Monte Carlo simulation studies.     

Conditional empirical likelihood for quantile regression models

In this paper, we propose a new Bayesian quantile regression estimator using conditional empirical likelihood as the working likelihood function. We show that the proposed estimator is asymptotically efficient and the confidence interval constructed is asymptotically valid. Our estimator has low computation cost since the posterior distribution function has explicit form. The finite sample performance of the proposed estimator is evaluated through Monte Carlo studies.     

Empirical likelihood for high-dimensional linear regression models

High-dimensional data are becoming prevalent, and many new methodologies and accompanying theories for high-dimensional data analysis have emerged in response. Empirical likelihood, as a classical nonparametric method of statistical inference, has proved to possess many good features. In this paper, our focus is to investigate the asymptotic behavior of empirical likelihood for regression coefficients in high-dimensional linear models. We give regularity conditions under which the standard normal calibration of empirical likelihood is valid in high dimensions. Both random and fixed designs are considered. Simulation studies are conducted to check the finite sample performance.     

Finite sample inference for quantile regression models

Under minimal assumptions, finite sample confidence bands for quantile regression models can be constructed. These confidence bands are based on the “conditional pivotal property” of estimating equations that quantile regression methods solve and provide valid finite sample inference for linear and nonlinear quantile models with endogenous or exogenous covariates. The confidence regions can be computed using Markov Chain Monte Carlo (MCMC) methods. We illustrate the finite sample procedure through two empirical examples: estimating a heterogeneous demand elasticity and estimating heterogeneous returns to schooling. We find pronounced differences between asymptotic and finite sample confidence regions in cases where the usual asymptotics are suspect.     

The limit distribution of the estimates in cointegrated regression models with multiple structural changes

We study estimation and inference in cointegrated regression models with multiple structural changes allowing both stationary and integrated regressors. Both pure and partial structural change models are analyzed. We derive the consistency, rate of convergence and the limit distribution of the estimated break fractions. Our technical conditions are considerably less restrictive than those in Bai et al. [Bai, J., Lumsdaine, R.L., Stock, J.H., 1998. Testing for and dating breaks in multivariate time series. Review of Economic Studies 65, 395–432] who considered the single break case in a multi-equations system, and permit a wide class of practically relevant models. Our analysis is, however, restricted to a single equation framework. We show that if the coefficients of the integrated regressors are allowed to change, the estimated break fractions are asymptotically dependent so that confidence intervals need to be constructed jointly. If, however, only the intercept and/or the coefficients of the stationary regressors are allowed to change, the estimates of the break dates are asymptotically independent as in the stationary case analyzed by Bai and Perron [Bai, J., Perron, P., 1998. Estimating and testing linear models with multiple structural changes. Econometrica 66, 47–78]. We also show that our results remain valid, under very weak conditions, when the potential endogeneity of the non-stationary regressors is accounted for via an increasing sequence of leads and lags of their first-differences as additional regressors. Simulation evidence is presented to assess the adequacy of the asymptotic approximations in finite samples.     

Quantile regression methods for recursive structural equation models

Two classes of quantile regression estimation methods for the recursive structural equation models of Chesher [2003. Identification in nonseparable models. Econometrica 71, 1405–1441.] are investigated. A class of weighted average derivative estimators based directly on the identification strategy of Chesher is contrasted with a new control variate estimation method. The latter imposes stronger restrictions achieving an asymptotic efficiency bound with respect to the former class. An application of the methods to the study of the effect of class size on the performance of Dutch primary school students shows that (i) reductions in class size are beneficial for good students in language and for weaker students in mathematics, (ii) larger classes appear beneficial for weaker language students, and (iii) the impact of class size on both mean and median performance is negligible.     

Exact permutation tests for non-nested non-linear regression models

This paper proposes exact distribution-free permutation tests for the specification of a non-linear regression model against one or more possibly non-nested alternatives. The new tests may be validly applied to a wide class of models, including models with endogenous regressors and lag structures. These tests build on the well-known J test developed by Davidson and MacKinnon [1981. Several tests for model specification in the presence of alternative hypotheses. Econometrica 49, 781–793] and their exactness holds under broader assumptions than those underlying the conventional J test. The J-type test statistics are used with a randomization or Monte Carlo resampling technique which yields an exact and computationally inexpensive inference procedure. A simulation experiment confirms the theoretical results and also shows the performance of the new procedure under violations of the maintained assumptions. The test procedure developed is illustrated by an application to inflation dynamics.     

On the backfitting algorithm for additive regression models

We analyse additive regression model fitting via the backfitting algorithm. We show that in the case of a large class of curve estimators, which includes regressograms, simple step-by-step formulae can be given for the back-fitting algorithm. The result of each cycle of the algorithm may be represented succinctly in terms of a sequence of d projections in n-dimensional space, where d is the number of design coordinates and n is sample size. It follows from our formulae that the limit of the algorithm is simply the projection of the data onto that vector space which is orthogonal to the space of all n-vectors fixed by each of the projections. The formulae also provide the convergence rate of the algorithm, the variance of the backfitting estimator, consistency of the estimator, and the relationship of the estimator to that obtained by directly minimizing mean squared distance.