Methods for Default and Robust Bayesian Model Comparison: the Fractional Bayes Factor Approach

In the Bayesian approach to model selection and hypothesis testing, the Bayes factor plays a central role. However, the Bayes factor is very sensitive to prior distributions of parameters. This is a problem especially in the presence of weak prior information on the parameters of the models. The most radical consequence of this fact is that the Bayes factor is undetermined when improper priors are used. Nonetheless, extending the non-informative approach of Bayesian analysis to model selection/testing procedures is important both from a theoretical and an applied viewpoint. The need to develop automatic and robust methods for model comparison has led to the introduction of several alternative Bayes factors. In this paper we review one of these methods: the fractional Bayes factor (O'Hagan, 1995). We discuss general properties of the method, such as consistency and coherence. Furthermore, in addition to the original, essentially asymptotic justifications of the fractional Bayes factor, we provide further finite-sample motivations for its use. Connections and comparisons to other automatic methods are discussed and several issues of robustness with respect to priors and data are considered. Finally, we focus on some open problems in the fractional Bayes factor approach, and outline some possible answers and directions for future research.     

Estimating the Kullback–Liebler risk based on multifold cross‐validation

This paper concerns a class of model selection criteria based on cross‐validation techniques and estimative predictive densities. Both the simple or leave‐one‐out and the multifold or leave‐m‐out cross‐validation procedures are considered. These cross‐validation criteria define suitable estimators for the expected Kullback–Liebler risk, which measures the expected discrepancy between the fitted candidate model and the true one. In particular, we shall investigate the potential bias of these estimators, under alternative asymptotic regimes for m. The results are obtained within the general context of independent, but not necessarily identically distributed, observations and by assuming that the candidate model may not contain the true distribution. An application to the class of normal regression models is also presented, and simulation results are obtained in order to gain some further understanding on the behavior of the estimators.     

Extrapolation‐based Bandwidth Selectors: A Review and Comparative Study with Discussion on Bivariate Applications

Cross‐validation is a widely used tool in selecting the smoothing parameter in a non‐parametric procedure. However, it suffers from large sampling variation and tends to overfit the data set. Many attempts have been made to reduce the variance of cross‐validation. This paper focuses on two recent proposals of extrapolation‐based cross‐validation bandwidth selectors: indirect cross‐validation and subsampling‐extrapolation technique. In univariate case, we notice that using a fixed value parameter surrogate for indirect cross‐validation works poorly when the true density is hard to estimate, while the subsampling‐extrapolation technique is more robust to non‐normality. We investigate whether a hybrid bandwidth selector could benefit from the advantages of both approaches and compare the performance of different extrapolation‐based bandwidth selectors through simulation studies, real data analyses and large sample theory. A discussion on their extension to bivariate case is also presented.     

Double Empirical Bayes Testing

Analysing data from large-scale, multiexperiment studies requires scientists to both analyse each experiment and to assess the results as a whole. In this article, we develop double empirical Bayes testing (DEBT), an empirical Bayes method for analysing multiexperiment studies when many covariates are gathered per experiment. DEBT is a two-stage method: in the first stage, it reports which experiments yielded significant outcomes and in the second stage, it hypothesises which covariates drive the experimental significance. In both of its stages, DEBT builds on the work of Efron, who laid out an elegant empirical Bayes approach to testing. DEBT enhances this framework by learning a series of black box predictive models to boost power and control the false discovery rate. In Stage 1, it uses a deep neural network prior to report which experiments yielded significant outcomes. In Stage 2, it uses an empirical Bayes version of the knockoff filter to select covariates that have significant predictive power of Stage 1 significance. In both simulated and real data, DEBT increases the proportion of discovered significant outcomes and selects more features when signals are weak. In a real study of cancer cell lines, DEBT selects a robust set of biologically plausible genomic drivers of drug sensitivity and resistance in cancer.     

Bayesian Calibration of p‐Values from Fisher's Exact Test

p‐Values are commonly transformed to lower bounds on Bayes factors, so‐called minimum Bayes factors. For the linear model, a sample‐size adjusted minimum Bayes factor over the class of g‐priors on the regression coefficients has recently been proposed (Held & Ott, The American Statistician 70(4), 335–341, 2016). Here, we extend this methodology to a logistic regression to obtain a sample‐size adjusted minimum Bayes factor for 2 × 2 contingency tables. We then study the relationship between this minimum Bayes factor and two‐sided p‐values from Fisher's exact test, as well as less conservative alternatives, with a novel parametric regression approach. It turns out that for all p‐values considered, the maximal evidence against the point null hypothesis is inversely related to the sample size. The same qualitative relationship is observed for minimum Bayes factors over the more general class of symmetric prior distributions. For the p‐values from Fisher's exact test, the minimum Bayes factors do on average not tend to the large‐sample bound as the sample size becomes large, but for the less conservative alternatives, the large‐sample behaviour is as expected.     

Asymptotics and smoothing parameter selection for penalized spline regression with various loss functions

Penalized splines are used in various types of regression analyses, including non‐parametric quantile, robust and the usual mean regression. In this paper, we focus on the penalized spline estimator with general convex loss functions. By specifying the loss function, we can obtain the mean estimator, quantile estimator and robust estimator. We will first study the asymptotic properties of penalized splines. Specifically, we will show the asymptotic bias and variance as well as the asymptotic normality of the estimator. Next, we will discuss smoothing parameter selection for the minimization of the mean integrated squares error. The new smoothing parameter can be expressed uniquely using the asymptotic bias and variance of the penalized spline estimator. To validate the new smoothing parameter selection method, we will provide a simulation. The simulation results show that the consistency of the estimator with the proposed smoothing parameter selection method can be confirmed and that the proposed estimator has better behavior than the estimator with generalized approximate cross‐validation. A real data example is also addressed.     

Model selection with estimated factors and idiosyncratic components

This paper provides consistent information criteria for the selection of forecasting models that use a subset of both the idiosyncratic and common factor components of a big dataset. This hybrid model approach has been explored by recent empirical studies to relax the strictness of pure factor‐augmented model approximations, but no formal model selection procedures have been developed. The main difference to previous factor‐augmented model selection procedures is that we must account for estimation error in the idiosyncratic component as well as the factors. Our main contribution is to show the conditions required for selection consistency of a class of information criteria that reflect this additional source of estimation error. We show that existing factor‐augmented model selection criteria are inconsistent in circumstances where N is of larger order than , where N and T are the cross‐section and time series dimensions of the dataset respectively, and that the standard Bayesian information criterion is inconsistent regardless of the relationship between N and T. We therefore propose a new set of information criteria that guarantee selection consistency in the presence of estimated idiosyncratic components. The properties of these new criteria are explored through a Monte Carlo simulation study. The paper concludes with an empirical application to long‐horizon exchange rate forecasting using a recently proposed model with country‐specific idiosyncratic components from a panel of global exchange rates.     

Generalized least squares cross‐validation in kernel density estimation

The kernel density estimation is a popular method in density estimation. The main issue is bandwidth selection, which is a well‐known topic and is still frustrating statisticians. A robust least squares cross‐validation bandwidth is proposed, which significantly improves the classical least squares cross‐validation bandwidth for its variability and undersmoothing, adapts to different kinds of densities, and outperforms the existing bandwidths in statistical literature and software.     

Bayesian Priors from Loss Matching

This paper is concerned with the construction of prior probability measures for parametric families of densities where the framework is such that only beliefs or knowledge about a single observable data point is required. We pay particular attention to the parameter which minimizes a measure of divergence to the distribution providing the data. The prior distribution reflects this attention and we discuss the application of the Bayes rule from this perspective. Our framework is fundamentally non‐parametric and we are able to interpret prior distributions on the parameter space using ideas of matching loss functions, one of which is coming from the data model and the other from the prior.     

Retirement Choices in Italy: What an Option Value Model Tells Us*

Using Italian data, we estimate an option value model to quantify the effect of financial incentives on retirement choices. As far as we know, this is the first empirical study to estimate the conditional multiple‐years model put forward by Stock and Wise (1990) . This implies that we account for dynamic self‐selection bias. We also present an extended version of this model in which the marginal value of leisure is random. For the female sample, the model is able to predict almost perfectly the age‐specific hazard rates. For the male sample, we obtain a good fit. Dynamic self‐selection results in a downward bias in the estimate of the marginal utility of leisure. We perform a simulation study to gauge the effects of a dramatic pension reform. Underestimation of the value of leisure translates into sizeable over‐prediction of the impact of reform. Due to lack of data, results for males should be interpreted with caution since we are not able to fully correct for dynamic self‐selection bias.     

Broad Cross‐License Negotiations

In many industries, broad cross‐license agreements are considered a useful method to obtain freedom to operate and to avoid patent litigation. In this paper, I study firm incentives to sign a broad cross‐license as well as the duration of broad cross‐license negotiations. I develop a model of bargaining with learning, which predicts that two firms will enter a broad cross‐license agreement only if their capital intensities are large enough. The model also predicts faster negotiations when firms have high capital intensities and when the frequency of future disputes is low. I confirm these predictions empirically using a novel data set on cross‐licensing and litigation in the US semiconductor industry.     

Robust Hierarchical Bayes Small Area Estimation for the Nested Error Linear Regression Model

Standard model‐based small area estimates perform poorly in presence of outliers. Sinha & Rao ( 2009 ) developed robust frequentist predictors of small area means. In this article, we present a robust Bayesian method to handle outliers in unit‐level data by extending the nested error regression model. We consider a finite mixture of normal distributions for the unit‐level error to model outliers and produce noninformative Bayes predictors of small area means. Our modelling approach generalises that of Datta & Ghosh ( 1991 ) under the normality assumption. Application of our method to a data set which is suspected to contain an outlier confirms this suspicion, correctly identifies the suspected outlier and produces robust predictors and posterior standard deviations of the small area means. Evaluation of several procedures including the M‐quantile method of Chambers & Tzavidis ( 2006 ) via simulations shows that our proposed method is as good as other procedures in terms of bias, variability and coverage probability of confidence and credible intervals when there are no outliers. In the presence of outliers, while our method and Sinha–Rao method perform similarly, they improve over the other methods. This superior performance of our procedure shows its dual (Bayes and frequentist) dominance, which should make it attractive to all practitioners, Bayesians and frequentists, of small area estimation.     

Do dropouts suffer from dropping out? Estimation and prediction of outcome gains in generalized selection models

In this paper we describe methods for predicting distributions of outcome gains in the framework of a latent variable selection model. We describe such procedures for Student‐t selection models and a finite mixture of Gaussian selection models. Importantly, our algorithms for fitting these models are simple to implement in practice, and also permit learning to take place about the non‐identified cross‐regime correlation parameter. Using data from High School and Beyond, we apply our methods to determine the impact of dropping out of high school on a math test score taken at the senior year of high school. Our results show that selection bias is an important feature of this data, that our beliefs about this non‐identified correlation are updated from the data, and that generalized models of selectivity offer an improvement over the ‘textbook’ Gaussian model. Further, our results indicate that on average dropping out of high school has a large negative impact on senior‐year test scores. However, for those individuals who actually drop out of high school, the act of dropping out of high school does not have a significantly negative impact on test scores. This suggests that policies aimed at keeping students in school may not be as beneficial as first thought, since those individuals who must be induced to stay in school are not the ones who benefit significantly (in terms of test scores) from staying in school. Copyright © 2004 John Wiley & Sons, Ltd.     

Efficient estimation of factor models with time and cross‐sectional dependence

This paper studies the efficient estimation of large‐dimensional factor models with both time and cross‐sectional dependence assuming (N,T) separability of the covariance matrix. The asymptotic distribution of the estimator of the factor and factor‐loading space under factor stationarity is derived and compared to that of the principal component (PC) estimator. The paper also considers the case when factors exhibit a unit root. We provide feasible estimators and show in a simulation study that they are more efficient than the PC estimator in finite samples. In application, the estimation procedure is employed to estimate the Lee–Carter model and life expectancy is forecast. The Dutch gender gap is explored and the relationship between life expectancy and the level of economic development is examined in a cross‐country comparison.     

What We can Learn About the Behaviour of Firms from the Average Monthly Frequency of Price‐Changes: An Application to the UK CPI Data

The monthly frequency of price‐changes is a prominent feature of many studies of the CPI micro‐data. In this paper, we see what the frequency implies for the behaviour of price‐setters in terms of the cross‐sectional distribution average of price‐spell durations across firms. We derive a lower bound for the mean duration of price‐spells averaged across firms. We use the UK CPI data at the aggregate and sectoral level and find that the actual mean is about twice the theoretical minimum consistent with the observed frequency. We construct hypothetical Bernoulli–Calvo distributions from the frequency data which we find can result in similar impulse responses to the estimated hazards when used in the Smets–Wouters (2003) model.     

Objective Bayes Factors for Inequality Constrained Hypotheses

This paper will present a Bayes factor for the comparison of an inequality constrained hypothesis with its complement or an unconstrained hypothesis. Equivalent sets of hypotheses form the basis for the quantification of the complexity of an inequality constrained hypothesis. It will be shown that the prior distribution can be chosen such that one of the terms in the Bayes factor is the quantification of the complexity of the hypothesis of interest. The other term in the Bayes factor represents a measure of the fit of the hypothesis. Using a vague prior distribution this fit value is essentially determined by the data. The result is an objective Bayes factor.     

基于情景判断的管理者胜任特征评价及效度研究

通过两次样本调查,研究了情景判断测验在测量管理者问题解决、人际胜任和伦理诚信等胜任特征方面的构思效度和效标关联效度。样本1运用验证型因素分析方法,通过替代模型策略,对测验的单维模型和多维模型进行比较,研究结果支持了多维模型假设,测验表现出较好的效标关联效度;在交叉效度检验研究中,测验存在一定程度的效度缩水,但仍表现出较好的效度水平。研究支持了胜任特征的情景判断测量,并对构思导向情景评价方法和意义进行了讨论。     

Tuning parameter selection in penalized generalized linear models for discrete data

In recent years, we have seen an increased interest in the penalized likelihood methodology, which can be efficiently used for shrinkage and selection purposes. This strategy can also result in unbiased, sparse, and continuous estimators. However, the performance of the penalized likelihood approach depends on the proper choice of the regularization parameter. Therefore, it is important to select it appropriately. To this end, the generalized cross‐validation method is commonly used. In this article, we firstly propose new estimates of the norm of the error in the generalized linear models framework, through the use of Kantorovich inequalities. Then these estimates are used in order to derive a tuning parameter selector in penalized generalized linear models. The proposed method does not depend on resampling as the standard methods and therefore results in a considerable gain in computational time while producing improved results. A thorough simulation study is conducted to support theoretical findings; and a comparison of the penalized methods with the L₁, the hard thresholding, and the smoothly clipped absolute deviation penalty functions is performed, for the cases of penalized Logistic regression and penalized Poisson regression. A real data example is being analyzed, and a discussion follows. © 2014 The Authors. Statistica Neerlandica © 2014 VVS.     

UK Multinationals' Effective Use of Financial Currency‐Hedge Techniques: Estimating and Explaining Foreign Exchange Exposure Using Bilateral Exchange Rates

Using a unique dataset of recently available accounting disclosures, this study examines the relationship between UK multinationals' stock returns and changes in the principal exchange rate to which each firm is most exposed. We find more firms with significant foreign exchange exposure estimates using this firm‐specific principal currency data, compared with those exposure estimates using the broad exchange rate index data prevalent in prior studies. The cross‐sectional variations in such principal‐currency exposure estimates are explained in relation to the financial currency‐hedge techniques that each firm specifically identifies as being used to manage its currency risk. In particular, we provide evidence that firms effectively use foreign currency derivatives and foreign‐denominated debt to reduce the currency risk associated with the bilateral exchange rate to which they are most exposed. This study is important to both the academic and the practitioner communities because it represents the first use of publicly available UK disclosures to improve the estimation and explanation of foreign exchange exposure.     

Cointegration Testing in Panels with Common Factors*

Panel unit‐root and no‐cointegration tests that rely on cross‐sectional independence of the panel unit experience severe size distortions when this assumption is violated, as has, for example, been shown by Banerjee, Marcellino and Osbat [Econometrics Journal (2004), Vol. 7, pp. 322–340; Empirical Economics (2005), Vol. 30, pp. 77–91] via Monte Carlo simulations. Several studies have recently addressed this issue for panel unit‐root tests using a common factor structure to model the cross‐sectional dependence, but not much work has been done yet for panel no‐cointegration tests. This paper proposes a model for panel no‐cointegration using an unobserved common factor structure, following the study by Bai and Ng [Econometrica (2004), Vol. 72, pp. 1127–1177] for panel unit roots. We distinguish two important cases: (i) the case when the non‐stationarity in the data is driven by a reduced number of common stochastic trends, and (ii) the case where we have common and idiosyncratic stochastic trends present in the data. We discuss the homogeneity restrictions on the cointegrating vectors resulting from the presence of common factor cointegration. Furthermore, we study the asymptotic behaviour of some existing residual‐based panel no‐cointegration tests, as suggested by Kao [Journal of Econometrics (1999), Vol. 90, pp. 1–44] and Pedroni [Econometric Theory (2004a), Vol. 20, pp. 597–625]. Under the data‐generating processes (DGP) used, the test statistics are no longer asymptotically normal, and convergence occurs at rate T rather than as for independent panels. We then examine the possibilities of testing for various forms of no‐cointegration by extracting the common factors and individual components from the observed data directly and then testing for no‐cointegration using residual‐based panel tests applied to the defactored data.