Nonparametric inference for counterfactual means: Bias-correction,confidence sets,and weak IV

This paper supplements Manski (1990) and Manski and Pepper (2000) and contributes to the literature by introducing the concept of weak IV for the partially identified mean counterfactual outcomes when an instrumental variable (IV) or a monotone instrumental variable (MIV) is available (IV or MIV assumption respectively); developing asymptotically uniformly valid confidence sets for the counterfactual mean outcomes and average treatment effects under the assumptions; correcting biases of estimates of bounds on the counterfactual mean outcomes under the assumptions. We apply the confidence sets to further examining the effect of family intactness on a child’s high school graduation originally studied in Manski et al. (1992).     

Conditional assessment of the impact of a Hausman pretest on confidence intervals

In the analysis of clustered and longitudinal data, which includes a covariate that varies both between and within clusters, a Hausman pretest is commonly used to decide whether subsequent inference is made using the linear random intercept model or the fixed effects model. We assess the effect of this pretest on the coverage probability and expected length of a confidence interval for the slope, conditional on the observed values of the covariate. This assessment has the advantages that it (i) relates to the values of this covariate at hand, (ii) is valid irrespective of how this covariate is generated, (iii) uses exact finite sample results, and (iv) results in an assessment that is determined by the values of this covariate and only two unknown parameters. For two real data sets, our conditional analysis shows that the confidence interval constructed after a Hausman pretest should not be used.     

Exact inference for the linear model with groupwise heteroscedastic spherical disturbances

Exact nonparametric inference on a single coefficient in a linear regression model, as considered by Bekker (Working Paper, Department of Economics, University of Groningen, 1997), is elaborated for the case of spherically distributed heteroscedastic disturbances. Instead of approximate inference based on feasible weighted least squares, exact inference is formulated based on partial rotational invariance of the distribution of the vector of disturbances. Thus, classical exact inference based on t-statistics is generalized to exact inference that remains valid in a groupwise heteroscedastic context. The approach is applied to a basic two-sample problem, and to the random- and fixed-effects models for panel data.     

Exact permutation tests for non-nested non-linear regression models

This paper proposes exact distribution-free permutation tests for the specification of a non-linear regression model against one or more possibly non-nested alternatives. The new tests may be validly applied to a wide class of models, including models with endogenous regressors and lag structures. These tests build on the well-known J test developed by Davidson and MacKinnon [1981. Several tests for model specification in the presence of alternative hypotheses. Econometrica 49, 781–793] and their exactness holds under broader assumptions than those underlying the conventional J test. The J-type test statistics are used with a randomization or Monte Carlo resampling technique which yields an exact and computationally inexpensive inference procedure. A simulation experiment confirms the theoretical results and also shows the performance of the new procedure under violations of the maintained assumptions. The test procedure developed is illustrated by an application to inflation dynamics.     

Finite sample inference for quantile regression models

Under minimal assumptions, finite sample confidence bands for quantile regression models can be constructed. These confidence bands are based on the “conditional pivotal property” of estimating equations that quantile regression methods solve and provide valid finite sample inference for linear and nonlinear quantile models with endogenous or exogenous covariates. The confidence regions can be computed using Markov Chain Monte Carlo (MCMC) methods. We illustrate the finite sample procedure through two empirical examples: estimating a heterogeneous demand elasticity and estimating heterogeneous returns to schooling. We find pronounced differences between asymptotic and finite sample confidence regions in cases where the usual asymptotics are suspect.     

The Coverage Properties of Confidence Regions After Model Selection

It is very common in applied frequentist ("classical") statistics to carry out a preliminary statistical (i.e. data-based) model selection by, for example, using preliminary hypothesis tests or minimizing AIC. This is usually followed by the inference of interest, using the same data, based on the assumption that the selected model had been given to us  a priori . This assumption is false and it can lead to an inaccurate and misleading inference. We consider the important case that the inference of interest is a confidence region. We review the literature that shows that the resulting confidence regions typically have very poor coverage properties. We also briefly review the closely related literature that describes the coverage properties of prediction intervals after preliminary statistical model selection. A possible motivation for preliminary statistical model selection is a wish to utilize uncertain prior information in the inference of interest. We review the literature in which the aim is to utilize uncertain prior information directly in the construction of confidence regions, without requiring the intermediate step of a preliminary statistical model selection. We also point out this aim as a future direction for research.     

A Review and Empirical Comparison of Bayesian and Classical Approaches to Inference on Efficiency Levels in Stochastic Frontier Models with Panel Data

This paper appliesa large number of models to three previously-analyzed data sets,and compares the point estimates and confidence intervals fortechnical efficiency levels. Classical procedures include multiplecomparisons with the best, based on the fixed effects estimates;a univariate version, marginal comparisons with the best; bootstrappingof the fixed effects estimates; and maximum likelihood givena distributional assumption. Bayesian procedures include a Bayesianversion of the fixed effects model, and various Bayesian modelswith informative priors for efficiencies. We find that fixedeffects models generally perform poorly; there is a large payoffto distributional assumptions for efficiencies. We do not findmuch difference between Bayesian and classical procedures, inthe sense that the classical MLE based on a distributional assumptionfor efficiencies gives results that are rather similar to a Bayesiananalysis with the corresponding prior.     

Identification robust confidence set methods for inference on parameter ratios with application to discrete choice models

We study the problem of building confidence sets for ratios of parameters, from an identification robust perspective. In particular, we address the simultaneous confidence set estimation of a finite number of ratios. Results apply to a wide class of models suitable for estimation by consistent asymptotically normal procedures. Conventional methods (e.g. the delta method) derived by excluding the parameter discontinuity regions entailed by the ratio functions and which typically yield bounded confidence limits, break down even if the sample size is large ( Dufour, 1997). One solution to this problem, which we take in this paper, is to use variants of  Fieller’s ( 1940, 1954) method. By inverting a joint test that does not require identifying the ratios, Fieller-based confidence regions are formed for the full set of ratios. Simultaneous confidence sets for individual ratios are then derived by applying projection techniques, which allow for possibly unbounded outcomes. In this paper, we provide simple explicit closed-form analytical solutions for projection-based simultaneous confidence sets, in the case of linear transformations of ratios. Our solution further provides a formal proof for the expressions in Zerbe et al. (1982) pertaining to individual ratios. We apply the geometry of quadrics as introduced by  and , in a different although related context. The confidence sets so obtained are exact if the inverted test statistic admits a tractable exact distribution, for instance in the normal linear regression context. The proposed procedures are applied and assessed via illustrative Monte Carlo and empirical examples, with a focus on discrete choice models estimated by exact or simulation-based maximum likelihood. Our results underscore the superiority of Fieller-based methods.     

Optimal inference for instrumental variables regression with non-Gaussian errors

This paper is concerned with inference on the coefficient on the endogenous regressor in a linear instrumental variables model with a single endogenous regressor, nonrandom exogenous regressors and instruments, and i.i.d. errors whose distribution is unknown. It is shown that under mild smoothness conditions on the error distribution it is possible to develop tests which are “nearly” efficient in the sense of Andrews et al. (2006) when identification is weak and consistent and asymptotically optimal when identification is strong. In addition, an estimator is presented which can be used in the usual way to construct valid (indeed, optimal) confidence intervals when identification is strong. The estimator is of the two stage least squares variety and is asymptotically efficient under strong identification whether or not the errors are normal.     

Confidence sets for the date of a break in level and trend when the order of integration is unknown

We propose methods for constructing confidence sets for the timing of a break in level and/or trend that have asymptotically correct coverage for both I(0) and I(1) processes. These are based on inverting a sequence of tests for the break location, evaluated across all possible break dates. We separately derive locally best invariant tests for the I(0) and I(1) cases; under their respective assumptions, the resulting confidence sets provide correct asymptotic coverage regardless of the magnitude of the break. We suggest use of a pre-test procedure to select between the I(0)- and I(1)-based confidence sets, and Monte Carlo evidence demonstrates that our recommended procedure achieves good finite sample properties in terms of coverage and length across both I(0) and I(1) environments. An application using US macroeconomic data is provided which further evinces the value of these procedures.     

Subsampling high frequency data

The main contribution of this paper is to propose a novel way of conducting inference for an important general class of estimators that includes many estimators of integrated volatility. A subsampling scheme is introduced that consistently estimates the asymptotic variance for an estimator, thereby facilitating inference and the construction of valid confidence intervals. The new method does not rely on the exact form of the asymptotic variance, which is useful when the latter is of complicated form. The method is applied to the volatility estimator of Aït-Sahalia et al. (2011) in the presence of autocorrelated and heteroscedastic market microstructure noise.     

Treatment effect bounds: An application to Swan–Ganz catheterization

We reanalyze data from the observational study by Connors et al. (1996) on the impact of Swan–Ganz catheterization on mortality outcomes. The study by Connors et al. (1996) assumes that there are no unobserved differences between patients who are catheterized and patients who are not catheterized and finds that catheterization increases patient mortality. We instead allow for such differences between patients by implementing both the instrumental variable bounds of Manski (1990), which only exploits an instrumental variable, and the bounds of Shaikh and Vytlacil (2011), which exploit mild nonparametric, structural assumptions in addition to an instrumental variable. We propose and justify the use of indicators of weekday admission as an instrument for catheterization in this context. We find that in our application, the Manski (1990) bounds do not indicate whether catheterization increases or decreases mortality, where as the Shaikh and Vytlacil (2011) bounds reveal that at least for some diagnoses, Swan–Ganz catheterization reduces mortality at 7 days after catheterization. We show that the bounds of Shaikh and Vytlacil (2011) remain valid under even weaker assumptions than those described in Shaikh and Vytlacil (2011). We also extend the analysis to exploit a further nonparametric, structural assumption–that doctors catheterize individuals with systematically worse latent health–and find that this assumption further narrows these bounds and strengthens our conclusions. In our analysis, we construct confidence regions using the methodology developed in Romano and Shaikh (2008). We show in particular that the confidence regions are uniformly consistent in level over a large class of possible distributions for the observed data that include distributions where the instrument is arbitrarily “weak”.     

Confidence Distribution,the Frequentist Distribution Estimator of a Parameter: A Review

In frequentist inference, we commonly use a single point (point estimator) or an interval (confidence interval/“interval estimator”) to estimate a parameter of interest. A very simple question is: Can we also use a distribution function (“distribution estimator”) to estimate a parameter of interest in frequentist inference in the style of a Bayesian posterior? The answer is affirmative, and confidence distribution is a natural choice of such a “distribution estimator”. The concept of a confidence distribution has a long history, and its interpretation has long been fused with fiducial inference. Historically, it has been misconstrued as a fiducial concept, and has not been fully developed in the frequentist framework. In recent years, confidence distribution has attracted a surge of renewed attention, and several developments have highlighted its promising potential as an effective inferential tool. This article reviews recent developments of confidence distributions, along with a modern definition and interpretation of the concept. It includes distributional inference based on confidence distributions and its extensions, optimality issues and their applications. Based on the new developments, the concept of a confidence distribution subsumes and unifies a wide range of examples, from regular parametric (fiducial distribution) examples to bootstrap distributions, significance (p‐value) functions, normalized likelihood functions, and, in some cases, Bayesian priors and posteriors. The discussion is entirely within the school of frequentist inference, with emphasis on applications providing useful statistical inference tools for problems where frequentist methods with good properties were previously unavailable or could not be easily obtained. Although it also draws attention to some of the differences and similarities among frequentist, fiducial and Bayesian approaches, the review is not intended to re‐open the philosophical debate that has lasted more than two hundred years. On the contrary, it is hoped that the article will help bridge the gaps between these different statistical procedures.     

The Multilevel Approach to Repeated Measures for Complete and Incomplete Data

Repeated measurements often are analyzed by multivariate analysis of variance (MANOVA). An alternative approach is provided by multilevel analysis, also called the hierarchical linear model (HLM), which makes use of random coefficient models. This paper is a tutorial which indicates that the HLM can be specified in many different ways, corresponding to different sets of assumptions about the covariance matrix of the repeated measurements. The possible assumptions range from the very restrictive compound symmetry model to the unrestricted multivariate model. Thus, the HLM can be used to steer a useful middle road between the two traditional methods for analyzing repeated measurements. Another important advantage of the multilevel approach to analyzing repeated measures is the fact that it can be easily used also if the data are incomplete. Thus it provides a way to achieve a fully multivariate analysis of repeated measures with incomplete data. This revised version was published online in June 2006 with corrections to the Cover Date.     

Ecological Fallacy and Covariates: New Insights based on Multilevel Modelling of Individual Data

The paper provides a new and more explicit formulation of the assumptions needed by the ordinary ecological regression to provide unbiased estimates and clarifies why violations of these assumptions will affect any method of ecological inference. Empirical evidence is obtained by showing that estimates provided by three main ecological inference methods are heavily biased when compared with multilevel logistic regression applied to a unique set of individual data on voting behaviour. The main findings of our paper have two important implications that can be extended to all situations where the assumptions needed to apply ecological inference are violated in the data: (i) only ecological inference methods that allow one to model the effect of covariates have a chance to produce unbiased estimates, and (ii) there are certain data generating mechanisms producing a kind of bias in ecological estimates that cannot be corrected by modelling the effect of covariates.     

Incorrect asymptotic size of subsampling procedures based on post-consistent model selection estimators

Subsampling and the m out of n bootstrap have been suggested in the literature as methods for carrying out inference based on post-model selection estimators and shrinkage estimators. In this paper we consider a subsampling confidence interval (CI) that is based on an estimator that can be viewed either as a post-model selection estimator that employs a consistent model selection procedure or as a super-efficient estimator. We show that the subsampling CI (of nominal level 1−α for any α(0,1)) has asymptotic confidence size (defined to be the limit of finite-sample size) equal to zero in a very simple regular model. The same result holds for the m out of n bootstrap provided m²/n→0 and the observations are i.i.d. Similar zero-asymptotic-confidence-size results hold in more complicated models that are covered by the general results given in the paper and for super-efficient and shrinkage estimators that are not post-model selection estimators. Based on these results, subsampling and the m out of n bootstrap are not recommended for obtaining inference based on post-consistent model selection or shrinkage estimators.     

Generalizing weak instrument robust IV statistics towards multiple parameters,unrestricted covariance matrices and identification statistics

We generalize the weak instrument robust score or Lagrange multiplier and likelihood ratio instrumental variables (IV) statistics towards multiple parameters and a general covariance matrix so they can be used in the generalized method of moments (GMM). The GMM extension of Moreira's [2003. A conditional likelihood ratio test for structural models. Econometrica 71, 1027–1048] conditional likelihood ratio statistic towards GMM preserves its expression except that it becomes conditional on a statistic that tests the rank of a matrix. We analyze the spurious power decline of Kleibergen's [2002. Pivotal statistics for testing structural parameters in instrumental variables regression. Econometrica 70, 1781–1803, 2005. Testing parameters in GMM without assuming that they are identified. Econometrica 73, 1103–1124] score statistic and show that an independent misspecification pre-test overcomes it. We construct identification statistics that reflect if the confidence sets of the parameters are bounded. A power study and the possible shapes of confidence sets illustrate the analysis.     

Statistical Inference under Symmetry

We explore the consequences of adjoining a symmetry group to a statistical model. Group actions are first induced on the sample space, and then on the parameter space. It is argued that the right invariant measure induced by the group on the parameter space is a natural non-informative prior for the parameters of the model. The permissible sub-parameters are introduced, i.e., the subparameters upon which group actions can be defined. Equivariant estimators are similarly defined. Orbits of the group are defined on the sample space and on the parameter space; in particular the group action is called transitive when there is only one orbit. Credibility sets and confidence sets are shown (under right invariant prior and assuming transitivity on the parameter space) to be equal when defined by permissible sub-parameters and constructed from equivariant estimators. The effect of different choices of transformation group is illustrated by examples, and properties of the orbits on the sample space and on the parameter space are discussed. It is argued that model reduction should be constrained to one or several orbits of the group. Using this and other natural criteria and concepts, among them concepts related to design of experiments under symmetry, leads to links towards chemometrical prediction methods and towards the foundation of quantum theory.     

A two-stage procedure for partially identified models

This paper studies a two-stage procedure for estimating partially identified models, based on Chernozhukov, Hong, and Tamer’s (2007) theory of set estimation and inference. We consider the case where a sub-vector of parameters or their identified set can be estimated separately from the rest, possibly subject to a priori restrictions. Our procedure constructs the second-stage set estimator and confidence set by taking appropriate level sets of a criterion function, using a first-stage estimator to impose restrictions on the parameter of interest. We give conditions under which the two-stage set estimator is a set-valued random element that is measurable in an appropriate sense. We also establish the consistency of the two-stage set estimator.