Measure for Measure: Exact F Tests and the Mixed Models Controversy

We consider exact F tests for the hypothesis of null random factor effect in the presence of interaction under the two factor mixed models involved in the mixed models controversy. We show that under the constrained parameter ( CP ) model, even in unbalanced data situations, MSB/MSE (in the usual ANOVA notation) follows an exact F distribution when the null hypothesis holds. We also obtain an exact F test for what is generally (and erroneously) assumed to be an equivalent hypothesis under the unconstrained parameter ( UP ) model. For unbalanced data, such a corresponding test statistic does not coincide with MSB/MSAB (the test statistic advocated for balanced data cases). We compute the power of the exact test under different imbalance patterns and show that although the loss of power increases with the degree of imbalance, it still remains reasonable from a practical point of view.