How to Choose a Working Model for Measuring the Statistical Evidence About a Regression Parameter

Consider using a likelihood ratio to measure the strength of statistical evidence for one hypothesis over another. Recent work has shown that when the model is correctly specified, the likelihood ratio is seldom misleading. But when the model is not, misleading evidence may be observed quite frequently. Here we consider how to choose a working regression model so that the statistical evidence is correctly represented as often as it would be under the true model. We argue that the criteria for choosing a working model should be how often it correctly represents the statistical evidence about the object of interest (regression coefficient in the true model). We see that misleading evidence about the object of interest is more likely to be observed when the working model is chosen according to other criteria (e.g., parsimony or predictive accuracy).