Bayesian model selection using encompassing priors

This paper deals with Bayesian selection of models that can be specified using inequality constraints among the model parameters. The concept of encompassing priors is introduced, that is, a prior distribution for an unconstrained model from which the prior distributions of the constrained models can be derived. It is shown that the Bayes factor for the encompassing and a constrained model has a very nice interpretation: it is the ratio of the proportion of the prior and posterior distribution of the encompassing model in agreement with the constrained model. It is also shown that, for a specific class of models, selection based on encompassing priors will render a virtually objective selection procedure. The paper concludes with three illustrative examples: an analysis of variance with ordered means; a contingency table analysis with ordered odds-ratios; and a multilevel model with ordered slopes.     

Methods for Default and Robust Bayesian Model Comparison: the Fractional Bayes Factor Approach

In the Bayesian approach to model selection and hypothesis testing, the Bayes factor plays a central role. However, the Bayes factor is very sensitive to prior distributions of parameters. This is a problem especially in the presence of weak prior information on the parameters of the models. The most radical consequence of this fact is that the Bayes factor is undetermined when improper priors are used. Nonetheless, extending the non-informative approach of Bayesian analysis to model selection/testing procedures is important both from a theoretical and an applied viewpoint. The need to develop automatic and robust methods for model comparison has led to the introduction of several alternative Bayes factors. In this paper we review one of these methods: the fractional Bayes factor (O'Hagan, 1995). We discuss general properties of the method, such as consistency and coherence. Furthermore, in addition to the original, essentially asymptotic justifications of the fractional Bayes factor, we provide further finite-sample motivations for its use. Connections and comparisons to other automatic methods are discussed and several issues of robustness with respect to priors and data are considered. Finally, we focus on some open problems in the fractional Bayes factor approach, and outline some possible answers and directions for future research.     

Exact inference in the inequality constrained normal linear regression model

Inference in the inequality constrained normal linear regression model is approached as a problem in Bayesian inference, using a prior that is the product of a conventional uninformative distribution and an indicator function representing the inequality constraints. The posterior distribution is calculated using Monte Carlo numerical integration, which leads directly to the evaluation of expected values of functions of interest. This approach is compared with others that have been proposed. Three empirical examples illustrate the utility of the proposed methods using an inexpensive 32-bit microcomputer.     

Design considerations for neyman-pearson and wald hypothesis testing

Summary The Neyman-Pearson Lemma describes a test for two simple hypotheses that, for a given sample size, is most powerful for its level. It is usually implemented by choosing the smallest sample size that achieves a prespecified power for a fixed level. The Lemma does not describe how to select either the level or the power of the test. In the usual Wald decision-theoretic structure there exists a sampling cost function, an initial prior over the hypothesis space and various payoffs to right/wrong hypothesis selections. The optimal Wald test is a Bayes decision rule that maximizes the expected payoff net of sampling costs. This paper shows that the Wald-optimal test and the Neyman-Pearson test can be the same and how the Neyman-Pearson test, with fixed level and power, can be viewed as a Wald test subject to restrictions on the payoff vector, cost function and prior distribution.     

Bayesian Calibration of p‐Values from Fisher's Exact Test

p‐Values are commonly transformed to lower bounds on Bayes factors, so‐called minimum Bayes factors. For the linear model, a sample‐size adjusted minimum Bayes factor over the class of g‐priors on the regression coefficients has recently been proposed (Held & Ott, The American Statistician 70(4), 335–341, 2016). Here, we extend this methodology to a logistic regression to obtain a sample‐size adjusted minimum Bayes factor for 2 × 2 contingency tables. We then study the relationship between this minimum Bayes factor and two‐sided p‐values from Fisher's exact test, as well as less conservative alternatives, with a novel parametric regression approach. It turns out that for all p‐values considered, the maximal evidence against the point null hypothesis is inversely related to the sample size. The same qualitative relationship is observed for minimum Bayes factors over the more general class of symmetric prior distributions. For the p‐values from Fisher's exact test, the minimum Bayes factors do on average not tend to the large‐sample bound as the sample size becomes large, but for the less conservative alternatives, the large‐sample behaviour is as expected.     

Empirical Bayes analysis of log-linear models for a generalized finite stationary Markov chain

This article presents the empirical Bayes method for estimation of the transition probabilities of a generalized finite stationary Markov chain whose ith state is a multi-way contingency table. We use a log-linear model to describe the relationship between factors in each state. The prior knowledge about the main effects and interactions will be described by a conjugate prior. Following the Bayesian paradigm, the Bayes and empirical Bayes estimators relative to various loss functions are obtained. These procedures are illustrated by a real example. Finally, asymptotic normality of the empirical Bayes estimators are established.     

Studentization and prediction in a multivariate normal setting

In a simple multivariate normal prediction setting, derivation of a predictive distribution can flow from formal Bayes arguments as well as pivoting arguments. We look at two special cases and show that the classical invariant predictive distribution is based on a pivot whose sampling distribution depends on the parameter – that is, the pivot is not an ancillary statistic. In contrast, a predictive distribution derived by a structural argument is based on a pivot with a parameter free distribution (an ancillary statistic). The classical procedure is formal Bayes for the Jeffreys prior. Our results show that this procedure does not have a structural or fiducial interpretation.     

Regional income inequality and economic development

It is the purpose of this paper to critically re-examine Williamson's original axiom of regional inequality as it relates to the process of national development under a new dimension: income distribution disparities within the population of the regions. The dynamic association between inequality and development in general and the convergence hypothesis in particular will be retested. Instead of Williamson's coefficient of variation we introduce a set of Regional Dissimilarity Indices that measures the dissimilarity of the spatial distribution of population by income at the national level.     

Small‐scale Inference: Empirical Bayes and Confidence Methods for as Few as a Single Comparison

Empirical Bayes methods of estimating the local false discovery rate (LFDR) by maximum likelihood estimation (MLE), originally developed for large numbers of comparisons, are applied to a single comparison. Specifically, when assuming a lower bound on the mixing proportion of true null hypotheses, the LFDR MLE can yield reliable hypothesis tests and confidence intervals given as few as one comparison. Simulations indicate that constrained LFDR MLEs perform markedly better than conventional methods, both in testing and in confidence intervals, for high values of the mixing proportion, but not for low values. (A decision‐theoretic interpretation of the confidence distribution made those comparisons possible.) In conclusion, the constrained LFDR estimators and the resulting effect‐size interval estimates are not only effective multiple comparison procedures but also they might replace p‐values and confidence intervals more generally. The new methodology is illustrated with the analysis of proteomics data.     

Measure for Measure: Exact F Tests and the Mixed Models Controversy

We consider exact F tests for the hypothesis of null random factor effect in the presence of interaction under the two factor mixed models involved in the mixed models controversy. We show that under the constrained parameter ( CP ) model, even in unbalanced data situations, MSB/MSE (in the usual ANOVA notation) follows an exact F distribution when the null hypothesis holds. We also obtain an exact F test for what is generally (and erroneously) assumed to be an equivalent hypothesis under the unconstrained parameter ( UP ) model. For unbalanced data, such a corresponding test statistic does not coincide with MSB/MSAB (the test statistic advocated for balanced data cases). We compute the power of the exact test under different imbalance patterns and show that although the loss of power increases with the degree of imbalance, it still remains reasonable from a practical point of view.     

On the natural selection rule in general linear models

We consider a problem of selecting the best treatment in a general linear model. We look at the properties of the natural selection rule. It is shown that the natural selection rule is minimax under to “0–1” loss function and it is a Bayes rule under a monotone permutation invariant loss function with respect to a permutation invariant prior for every variance balanced design. Some other condition on the design matrix is given so that a Bayes rule with respect to a normal prior will be of simple structure.     

Bayes Convolution

A general convolution theorem within a Bayesian framework is presented. Consider estimation of the Euclidean parameter θ by an estimator T within a parametric model. Let W be a prior distribution for θ and define G as the W -average of the distribution of T - θ under θ . In some cases, for any estimator T the distribution G can be written as a convolution G = K * L with K a distribution depending only on the model, i.e. on W and the distributions under θ of the observations. In such a Bayes convolution result optimal estimators exist, satisfying G = K . For location models we show that finite sample Bayes convolution results hold in the normal, loggamma and exponential case. Under regularity conditions we prove that normal and loggamma are the only smooth location cases. We also discuss relations with classical convolution theorems.     

Data‐Driven Identification Constraints for DSGE Models

We propose imposing data‐driven identification constraints to alleviate the multimodality problem arising in the estimation of poorly identified dynamic stochastic general equilibrium models under non‐informative prior distributions. We also devise an iterative procedure based on the posterior density of the parameters for finding these constraints. An empirical application to the Smets and Wouters ( 2007 ) model demonstrates the properties of the estimation method, and shows how the problem of multimodal posterior distributions caused by parameter redundancy is eliminated by identification constraints. Out‐of‐sample forecast comparisons as well as Bayes factors lend support to the constrained model.     

Bayesian Priors from Loss Matching

This paper is concerned with the construction of prior probability measures for parametric families of densities where the framework is such that only beliefs or knowledge about a single observable data point is required. We pay particular attention to the parameter which minimizes a measure of divergence to the distribution providing the data. The prior distribution reflects this attention and we discuss the application of the Bayes rule from this perspective. Our framework is fundamentally non‐parametric and we are able to interpret prior distributions on the parameter space using ideas of matching loss functions, one of which is coming from the data model and the other from the prior.     

Optimal hypothesis testing: from semi to fully Bayes factors

We propose and examine statistical test-strategies that are somewhat between the maximum likelihood ratio and Bayes factor methods that are well addressed in the literature. The paper shows an optimality of the proposed tests of hypothesis. We demonstrate that our approach can be easily applied to practical studies, because execution of the tests does not require deriving of asymptotical analytical solutions regarding the type I error. However, when the proposed method is utilized, the classical significance level of tests can be controlled.     

Bayesian hypothesis testing in latent variable models

Hypothesis testing using Bayes factors (BFs) is known not to be well defined under the improper prior. In the context of latent variable models, an additional problem with BFs is that they are difficult to compute. In this paper, a new Bayesian method, based on the decision theory and the EM algorithm, is introduced to test a point hypothesis in latent variable models. The new statistic is a by-product of the Bayesian MCMC output and, hence, easy to compute. It is shown that the new statistic is appropriately defined under improper priors because the method employs a continuous loss function. In addition, it is easy to interpret. The method is illustrated using a one-factor asset pricing model and a stochastic volatility model with jumps.     

Testing log-linear models with inequality constraints: a comparison of asymptotic, bootstrap, and posterior predictive p-values

An important aspect of applied research is the assessment of the goodness-of-fit of an estimated statistical model. In the analysis of contingency tables, this usually involves determining the discrepancy between observed and estimated frequencies using the likelihood-ratio statistic. In models with inequality constraints, however, the asymptotic distribution of this statistic depends on the unknown model parameters and, as a result, there no longer exists an unique p -value. Bootstrap p -values obtained by replacing the unknown parameters by their maximum likelihood estimates may also be inaccurate, especially if many of the imposed inequality constraints are violated in the available sample. We describe the various problems associated with the use of asymptotic and bootstrap p -values and propose the use of Bayesian posterior predictive checks as a better alternative for assessing the fit of log-linear models with inequality constraints.     

Minimax‐Optimal Strength of Statistical Evidence for a Composite Alternative Hypothesis

While the likelihood ratio measures statistical support for an alternative hypothesis about a single parameter value, it is undefined for an alternative hypothesis that is composite in the sense that it corresponds to multiple parameter values. Regarding the parameter of interest as a random variable enables measuring support for a composite alternative hypothesis without requiring the elicitation or estimation of a prior distribution, as described below. In this setting, in which parameter randomness represents variability rather than uncertainty, the ideal measure of the support for one hypothesis over another is the difference in the posterior and prior log‐odds. That ideal support may be replaced by any measure of support that, on a per‐observation basis, is asymptotically unbiased as a predictor of the ideal support. Such measures of support are easily interpreted and, if desired, can be combined with any specified or estimated prior probability of the null hypothesis. Two qualifying measures of support are minimax‐optimal. An application to proteomics data indicates that a modification of optimal support computed from data for a single protein can closely approximate the estimated difference in posterior and prior odds that would be available with the data for 20 proteins.     

Prior‐free Bayes Factors Based on Data Splitting

Bayes factors that do not require prior distributions are proposed for testing one parametric model versus another. These Bayes factors are relatively simple to compute, relying only on maximum likelihood estimates, and are Bayes consistent at an exponential rate for nested models even when the smaller model is true. These desirable properties derive from the use of data splitting. Large sample properties, including consistency, of the Bayes factors are derived, and a simulation study explores practical concerns. The methodology is illustrated with civil engineering data involving compressive strength of concrete.     

Empiric bayes estimation of hypergeometric probability

A Bayes-empiric Bayes estimator of a parameter of the hypergeometric distribution, based on orthogonal polynomials on non-negative integers, is introduced. It is shown that this estimator is asymptotically optimal; and the resulting estimator of the prior probability function is mean square consistent.