Comparison of Hypothesis Testing and Bayesian Model Selection

The main goal of both Bayesian model selection and classical hypotheses testing is to make inferences with respect to the state of affairs in a population of interest. The main differences between both approaches are the explicit use of prior information by Bayesians, and the explicit use of null distributions by the classicists. Formalization of prior information in prior distributions is often difficult. In this paper two practical approaches (encompassing priors and training data) to specify prior distributions will be presented. The computation of null distributions is relatively easy. However, as will be illustrated, a straightforward interpretation of the resulting p-values is not always easy. Bayesian model selection can be used to compute posterior probabilities for each of a number of competing models. This provides an alternative for the currently prevalent testing of hypotheses using p-values. Both approaches will be compared and illustrated using case studies. Each case study fits in the framework of the normal linear model, that is, analysis of variance and multiple regression.     

Bayesian Model Selection with an Uninformative Prior*

Bayesian model selection with posterior probabilities and no subjective prior information is generally not possible because of the Bayes factors being ill‐defined. Using careful consideration of the parameter of interest in cointegration analysis and a re‐specification of the triangular model of Phillips (Econometrica, Vol. 59, pp. 283–306, 1991), this paper presents an approach that allows for Bayesian comparison of models of cointegration with ‘ignorance’ priors. Using the concept of Stiefel and Grassman manifolds, diffuse priors are specified on the dimension and direction of the cointegrating space. The approach is illustrated using a simple term structure of the interest rates model.     

Bayesian Model Selection of Gaussian Directed Acyclic Graph Structures

During the last years, graphical models have become a popular tool to represent dependencies among variables in many scientific areas. Typically, the objective is to discover dependence relationships that can be represented through a directed acyclic graph (DAG). The set of all conditional independencies encoded by a DAG determines its Markov property. In general, DAGs encoding the same conditional independencies are not distinguishable from observational data and can be collected into equivalence classes, each one represented by a chain graph called essential graph (EG). However, both the DAG and EG space grow super exponentially in the number of variables, and so, graph structural learning requires the adoption of Markov chain Monte Carlo (MCMC) techniques. In this paper, we review some recent results on Bayesian model selection of Gaussian DAG models under a unified framework. These results are based on closed-form expressions for the marginal likelihood of a DAG and EG structure, which is obtained from a few suitable assumptions on the prior for model parameters. We then introduce a general MCMC scheme that can be adopted both for model selection of DAGs and EGs together with a couple of applications on real data sets.     

Statistical Decision Problems and Bayesian Nonparametric Methods

This paper considers parametric statistical decision problems conducted within a Bayesian nonparametric context. Our work was motivated by the realisation that typical parametric model selection procedures are essentially incoherent. We argue that one solution to this problem is to use a flexible enough model in the first place, a model that will not be checked no matter what data arrive. Ideally, one would use a nonparametric model to describe all the uncertainty about the density function generating the data. However, parametric models are the preferred choice for many statisticians, despite the incoherence involved in model checking, incoherence that is quite often ignored for pragmatic reasons. In this paper we show how coherent parametric inference can be carried out via decision theory and Bayesian nonparametrics. None of the ingredients discussed here are new, but our main point only becomes evident when one sees all priors—even parametric ones—as measures on sets of densities as opposed to measures on finite-dimensional parameter spaces.     

Methods and Tools for Bayesian Variable Selection and Model Averaging in Normal Linear Regression

In this paper, we briefly review the main methodological aspects concerned with the application of the Bayesian approach to model choice and model averaging in the context of variable selection in regression models. This includes prior elicitation, summaries of the posterior distribution and computational strategies. We then examine and compare various publicly available R‐packages, summarizing and explaining the differences between packages and giving recommendations for applied users. We find that all packages reviewed (can) lead to very similar results, but there are potentially important differences in flexibility and efficiency of the packages.     

A Non-Iterative Bayesian Approach to Statistical Matching

Data fusion or statistical matching techniques merge datasets from different survey samples to achieve a complete but artificial data file which contains all variables of interest. The merging of datasets is usually done on the basis of variables common to all files, but traditional methods implicitly assume conditional independence between the variables never jointly observed given the common variables. Therefore we suggest using model based approaches tackling the data fusion task by more flexible procedures. By means of suitable multiple imputation techniques, the identification problem which is inherent in statistical matching is reflected. Here a non-iterative Bayesian version of Rubin's implicit regression model is presented and compared in a simulation study with imputations from a data augmentation algorithm as well as an iterative approach using chained equations.     

Bayesian Inference in Spatial Sample Selection Models

In this study, we consider Bayesian methods for the estimation of a sample selection model with spatially correlated disturbance terms. We design a set of Markov chain Monte Carlo algorithms based on the method of data augmentation. The natural parameterization for the covariance structure of our model involves an unidentified parameter that complicates posterior analysis. The unidentified parameter – the variance of the disturbance term in the selection equation – is handled in different ways in these algorithms to achieve identification for other parameters. The Bayesian estimator based on these algorithms can account for the selection bias and the full covariance structure implied by the spatial correlation. We illustrate the implementation of these algorithms through a simulation study and an empirical application.     

Bayesian Variable Selection in Spatial Autoregressive Models

This paper compares the performance of Bayesian variable selection approaches for spatial autoregressive models. It presents two alternative approaches that can be implemented using Gibbs sampling methods in a straightforward way and which allow one to deal with the problem of model uncertainty in spatial autoregressive models in a flexible and computationally efficient way. A simulation study shows that the variable selection approaches tend to outperform existing Bayesian model averaging techniques in terms of both in-sample predictive performance and computational efficiency. The alternative approaches are compared in an empirical application using data on economic growth for European NUTS-2 regions.     

Bayesian Model Averaging and exchange rate forecasts

Exchange rate forecasting is hard and the seminal result of Meese and Rogoff [Meese, R., Rogoff, K., 1983. Empirical exchange rate models of the seventies: Do they fit out of sample? Journal of International Economics 14, 3–24] that the exchange rate is well approximated by a driftless random walk, at least for prediction purposes, still stands despite much effort at constructing other forecasting models. However, in several other macro and financial forecasting applications, researchers in recent years have considered methods for forecasting that effectively combine the information in a large number of time series. In this paper, I apply one such method for pooling forecasts from several different models, Bayesian Model Averaging, to the problem of pseudo out-of-sample exchange rate predictions. For most currency–horizon pairs, the Bayesian Model Averaging forecasts using a sufficiently high degree of shrinkage, give slightly smaller out-of-sample mean square prediction error than the random walk benchmark. The forecasts generated by this model averaging methodology are however very close to, but not identical to, those from the random walk forecast.     

Outlier‐Robust Bayesian Multinomial Choice Modeling

A Bayesian method for outlier‐robust estimation of multinomial choice models is presented. The method can be used for both correlated as well as uncorrelated choice alternatives and guarantees robustness towards outliers in the dependent and independent variables. To account for outliers in the response direction, the fat‐tailed multivariate Laplace distribution is used. Leverage points are handled via a shrinkage procedure. A simulation study shows that estimation of the model parameters is less influenced by outliers compared to non‐robust alternatives. An analysis of margarine scanner data shows how our method can be used for better pricing decisions. Copyright © 2015 John Wiley & Sons, Ltd.     

Bayesian Estimation of Prediction Error and Variable Selection in Linear Regression

An important statistical application is the problem of determining an appropriate set of input variables for modelling a response variable. In such an application, candidate models are characterized by which input variables are included in the mean structure. A reasonable approach to gauging the propriety of a candidate model is to define a discrepancy function through the prediction error associated with this model. An optimal set of input variables is then determined by searching for the candidate model that minimizes the prediction error. In this paper, we focus on a Bayesian approach to estimating a discrepancy function based on prediction error in linear regression. It is shown how this approach provides an informative method for quantifying model selection uncertainty.