Algorithms for optimal design with application to multiple polynomial regression

The approximate theory of optimal linear regression design leads to specific convex extremum problems for numerical solution. A conceptual algorithm is stated, whose concrete versions lead us from steepest descent type algorithms to improved gradient methods, and finally to second order methods with excellent convergence behaviour. Applications are given to symmetric multiple polynomial models of degree three or less, where invariance structures are utilized. A final section is devoted to the construction of efficientexact designs of sizeN from the optimal approximate designs. For the multifactor cubic model and some of the most popular optimality criteria (D-, A-, andI-criteria) fairly efficient exact designs are obtained, even for small sample sizeN. AMS Subject Classification: 62K05.Abbreviated Title: Algorithms for Optimal Design.Invited paper presented at the International Conference on Mathematical Statistics,ProbaStat '94, Smolenice, Slovakia.     

Local polynomial estimation of nonparametric simultaneous equations models

We define a new procedure for consistent estimation of nonparametric simultaneous equations models under the conditional mean independence restriction of Newey et al. [1999. Nonparametric estimation of triangular simultaneous equation models. Econometrica 67, 565–603]. It is based upon local polynomial regression and marginal integration techniques. We establish the asymptotic distribution of our estimator under weak data dependence conditions. Simulation evidence suggests that our estimator may significantly outperform the estimators of Pinkse [2000. Nonparametric two-step regression estimation when regressors and errors are dependent. Canadian Journal of Statistics 28, 289–300] and Newey and Powell [2003. Instrumental variable estimation of nonparametric models. Econometrica 71, 1565–1578].     

Efficient discriminating design for a class of nested polynomial regression models

This paper studies efficient designs for simultaneous model discrimination among polynomial regression models up to degree k. Based on the -optimality criterion proposed by Dette (Ann Stat 22:890–903, 1994), a maximin -optimal discriminating design is derived in terms of canonical moments for . Theoretical and numerical results show that the proposed design performs well for model discrimination in most of the considered models.     

Threshold estimation for continuous three-phase polynomial regression models with constant mean in the middle regime

This paper considers a continuous three-phase polynomial regression model with two threshold points for dependent data with heteroscedasticity. We assume the model is polynomial of order zero in the middle regime, and is polynomial of higher orders elsewhere. We denote this model by ${ℳ}_2$, which includes models with one or no threshold points, denoted by ${ℳ}_1$ and ${ℳ}_0$, respectively, as special cases. We provide an ordered iterative least squares (OiLS) method when estimating ${ℳ}_2$ and establish the consistency of the OiLS estimators under mild conditions. When the underlying model is ${ℳ}_1$ and is $(d_0-1)$th-order differentiable but not $d_0$th-order differentiable at the threshold point, we further show the $O_p(N^{-1/(d_0+2)})$ convergence rate of the OiLS estimators, which can be faster than the $O_p(N^{-1/(2d_0)})$ convergence rate given in Feder when $d_0\ge 3$. We also apply a model-selection procedure for selecting ${ℳ}_{\kappa }$; $\kappa =0,1,2$. When the underlying model exists, we establish the selection consistency under the aforementioned conditions. Finally, we conduct simulation experiments to demonstrate the finite-sample performance of our asymptotic results.     

Delay times of sequential procedures for multiple time series regression models

We consider a multiple regression model in which the explanatory variables are specified by time series. To sequentially test for the stability of the regression parameters in time, we introduce a detector which is based on the first excess time of a CUSUM-type statistic over a suitably constructed threshold function. The aim of this paper is to study the delay time associated with this detector. As our main result, we derive the limit distribution of the delay time and provide thereby a theory that extends the benchmark average run-length concept utilized in most of the sequential monitoring literature. To highlight the applicability of the limit results in finite samples, we present a Monte Carlo simulation study and an application to macroeconomic data.     

Sequential fixed-width confidence bands for distribution functions under random censoring

Summary Sequential fixed-width confidence bands for a distribution function are derived in the case, when the data are censored from the right. The Breslow-Crowley invariance principle for the Kaplan Meier estimate is extended to the random sample size situation. Also some simulation results are reported, which illustrate the behavior of the stopping times.     

Uniform confidence bands for functions estimated nonparametrically with instrumental variables

This paper is concerned with developing uniform confidence bands for functions estimated nonparametrically with instrumental variables. We show that a sieve nonparametric instrumental variables estimator is pointwise asymptotically normally distributed. The asymptotic normality result holds in both mildly and severely ill-posed cases. We present methods to obtain a uniform confidence band and show that the bootstrap can be used to obtain the required critical values. Monte Carlo experiments illustrate the finite-sample performance of the uniform confidence band.     

Selecting local models in multiple regression by maximizing power

This paper considers multiple regression procedures for analyzing the relationship between a response variable and a vector of d covariates in a nonparametric setting where tuning parameters need to be selected. We introduce an approach which handles the dilemma that with high dimensional data the sparsity of data in regions of the sample space makes estimation of nonparametric curves and surfaces virtually impossible. This is accomplished by abandoning the goal of trying to estimate true underlying curves and instead estimating measures of dependence that can determine important relationships between variables. These dependence measures are based on local parametric fits on subsets of the covariate space that vary in both dimension and size within each dimension. The subset which maximizes a signal to noise ratio is chosen, where the signal is a local estimate of a dependence parameter which depends on the subset dimension and size, and the noise is an estimate of the standard error (SE) of the estimated signal. This approach of choosing the window size to maximize a signal to noise ratio lifts the curse of dimensionality because for regions with sparsity of data the SE is very large. It corresponds to asymptotically maximizing the probability of correctly finding nonspurious relationships between covariates and a response or, more precisely, maximizing asymptotic power among a class of asymptotic level αt-tests indexed by subsets of the covariate space. Subsets that achieve this goal are called features. We investigate the properties of specific procedures based on the preceding ideas using asymptotic theory and Monte Carlo simulations and find that within a selected dimension, the volume of the optimally selected subset does not tend to zero as n → ∞ unless the volume of the subset of the covariate space where the response depends on the covariate vector tends to zero.     

Exact and approximate D-optimal designs in polynomial regression

If the sample sizen is large enough, then the exact polynomial regression designs obtained by rounding the weights of the approximate D-optimal design to integral multiples of 1/n are D-optimal. This was shown by alaevskiî (1966) and Gaffke (1987). In this note, an efficient algorithm to determine the minimum sample sizen _d for a polynomial model of degreed is derived from a condition given by Huang (1987). Under an additional assumption we show that the conditions of Gaffke and Huang are equivalent; we verify the additional assumption for polynomial degreed40.     

Multivariate regression models for panel data

The paper examines the relationship between heterogeneity bias and strict exogeneity in a distributed lag regression of y on x. The relationship is very strong when x is continuous, weaker when x is discrete, and non-existent as the order of the distributed lag becomes infinite. The individual specific random variables introduce nonlinearity and heteroskedasticity; so the paper provides an appropriate framework for the estimation of multivariate linear predictors. Restrictions are imposed using a minimum distance estimator. It is generally more efficient than the conventional estimators such as quasi-maximum likelihood. There are computationally simple generalizations of two- and three-stage least squares that achieve this efficiency gain. Some of these ideas are illustrated using the sample of Young Men in the National Longitudinal Survey. The paper reports regressions on the leads and lags of variables measuring union coverage, SMSA, and region. The results indicate that the leads and lags could have been generated just by a random intercept. This gives some support for analysis of covariance type estimates; these estimates indicate a substantial heterogeneity bias in the union, SMSA, and region coefficients.     

Identification and estimation of polynomial errors-in-variables models

Methods of estimation of regression coefficients are proposed when the regression function includes a polynomial in a ‘true’ regressor which is measured with error. Two sources of additional information concerning the unobservable regressor are considered: either an additional indicator of the regressor (itself measured with error) or instrumental variables which characterize the systematic variation in the true regressor. In both cases, estimators are constructed by relating moments involving the unobserved variables to moments of observables; these relations lead to recursion formulae for computation of the regression coefficients and nuisance parameters (e.g., moments of the measurement error). Consistency and asymptotic normality of the estimated coefficients is demonstrated, and consistent estimators of the asymptotic covariant matrices are provided.     

Optimal and robust designs for trigonometric regression models

This article presents discussions on the optimal and robust designs for trigonometric regression models under different optimality criteria. First, we investigate the classical Q-optimal designs for estimating the response function in a full trigonometric regression model with a given order. The equivalencies of Q-, A-, and G-optimal designs for trigonometric regression in general are also articulated. Second, we study minimax designs and their implementation in the case of trigonometric approximation under Q-, A-, and D-optimality. Then, We indicate the existence of the symmetric designs that are D-optimal minimax designs for general trigonometric regression models, and prove the existence of the symmetric designs that are Q- or A-optimal minimax designs for two particular trigonometric regression models under certain conditions.     

A consistent estimator for nonlinear regression models

In this paper an estimator for the general (nonlinear) regression model with random regressors is studied which is based on the Fourier transform of a certain weight function. Consistency and asymptotic normality of the estimator are established and simulation results are presented to illustrate the theoretical ones.Supported by the Hungarian National Science Foundation OTKA under Grants No. F 032060/2000 and F 046061/2004 and by the Bolyai Grant of the Hungarian Academy of Sciences.Received October 2003     

Useful invariance results for generalized regression models

Certain exact finite-sample invariance results are established for the usual estimators and test statistics in the generalized regression model with non-scalar covariance matrix. By inferring properties of an estimator from the criterion function which defines it, results can be obtained even when there is no explicit solution for the estimator as a function of the data (as, e.g., with maximum likelihood). Applications are illustrated by an examination of some recently published Monte Carlo simulation studies.     

Finite sample inference for quantile regression models

Under minimal assumptions, finite sample confidence bands for quantile regression models can be constructed. These confidence bands are based on the “conditional pivotal property” of estimating equations that quantile regression methods solve and provide valid finite sample inference for linear and nonlinear quantile models with endogenous or exogenous covariates. The confidence regions can be computed using Markov Chain Monte Carlo (MCMC) methods. We illustrate the finite sample procedure through two empirical examples: estimating a heterogeneous demand elasticity and estimating heterogeneous returns to schooling. We find pronounced differences between asymptotic and finite sample confidence regions in cases where the usual asymptotics are suspect.     

Empirical likelihood for high-dimensional linear regression models

High-dimensional data are becoming prevalent, and many new methodologies and accompanying theories for high-dimensional data analysis have emerged in response. Empirical likelihood, as a classical nonparametric method of statistical inference, has proved to possess many good features. In this paper, our focus is to investigate the asymptotic behavior of empirical likelihood for regression coefficients in high-dimensional linear models. We give regularity conditions under which the standard normal calibration of empirical likelihood is valid in high dimensions. Both random and fixed designs are considered. Simulation studies are conducted to check the finite sample performance.     

Conditional empirical likelihood for quantile regression models

In this paper, we propose a new Bayesian quantile regression estimator using conditional empirical likelihood as the working likelihood function. We show that the proposed estimator is asymptotically efficient and the confidence interval constructed is asymptotically valid. Our estimator has low computation cost since the posterior distribution function has explicit form. The finite sample performance of the proposed estimator is evaluated through Monte Carlo studies.     

Simultaneous confidence bands: Theory,implementation, and an application to SVARs

Simultaneous confidence bands are versatile tools for visualizing estimation uncertainty for parameter vectors, such as impulse response functions. In linear models, it is known that that the sup‐t confidence band is narrower than commonly used alternatives—for example, Bonferroni and projection bands. We show that the same ranking applies asymptotically even in general nonlinear models, such as vector autoregressions (VARs). Moreover, we provide further justification for the sup‐t band by showing that it is the optimal default choice when the researcher does not know the audience's preferences. Complementing existing plug‐in and bootstrap implementations, we propose a computationally convenient Bayesian sup‐t band with exact finite‐sample simultaneous credibility. In an application to structural VAR impulse response function estimation, the sup‐t band—which has been surprisingly overlooked in this setting—is at least 35% narrower than other off‐the‐shelf simultaneous bands.     

How to implement the bootstrap in static or stable dynamic regression models: test statistic versus confidence region approach

By combining two alternative formulations of a test statistic with two alternative resampling schemes we obtain four different bootstrap tests. In the context of static linear regression models two of these are shown to have serious size and power problems, whereas the remaining two are adequate and in fact equivalent. The equivalence between the two valid implementations is shown to break down in dynamic regression models. Then, the procedure based on the test statistic approach performs best, at least in the AR(1)-model. Similar finite-sample phenomena are illustrated in the ARMA(1,1)-model through a small-scale Monte Carlo study and an empirical example.