Influence of equal or unequal comparison group sample sizes on the detection of differential item functioning using the Mantel–Haenszel and logistic regression techniques

In recent decades several methods have been developed for detecting differential item functioning (DIF), and many studies have aimed to identify both the conditions under which items may or may not be adequate and the factors which affect their power and Type I error. This paper describes a Monte Carlo experiment that was carried out in order to analyse the effect of reference group sample size, focal group sample size and the interaction of the two on the power and Type I error of the Mantel–Haenszel (MH) and Logistic regression (LR) procedures. The data were generated using a three-parameter logistic model, the design was fully-crossed factorial with 12 experimental conditions arising from the crossing of the two main factors, and the dependent variables were power and the rate of false positives calculated across 100 replications. The results enabled the significant factors to be identified and the two statistics to be compared. Practical recommendations are made regarding use of the procedures by psychologists interested in the development and analysis of psychological tests.     

Nonuniform DIF Detection using Discriminant Logistic Analysis and Multinomial Logistic Regression: A comparison for polytomous items

This study focused on the effectiveness in nonuniform polytomous item DIF detection using Discriminant Logistic Analysis (DLA) and Multinomial Logistic Regression (MLR). A computer simulation study was conducted to compare the effect of using DLA and MLR, applying either an iterative test purification procedure or non-iterative to detect nonuniform DIF. The conditions under study were: DIF effect size (0.5, 1.0 and 1.5), sample size (500 and 1000), percentage of DIF items in the test (0, 10 and 20%) and DIF type (nonuniform). The results suggest that DLA is more accurate than MLR in detecting DIF. However, the purification process only improved the correct detection rate when MLR was applied. The false positive rates for both procedures were similar. Moreover, when the test purification procedure was used, the proportion of non-DIF items that were detected as DIF decreased for both procedures, although the false positive rates were smaller for DLA than for MLR.     

The Performance of Some Observed and Unobserved Conditional Invariance Techniques for the Detection of Differential Item Functioning

The current research investigates the performance of some conditional and unconditional invariance measures of differential item functioning (DIF), namely: the log-linear model, the logistic regression model, the signed and the unsigned area, the SOS1 and the SOS3. A simulation study is used to evaluate their ability to detect uniform as well as nondirectional nonuniform DIF under several test conditions. The factors that were subject to experimental manipulation in the simulation study are the size of the DIF, the ability distribution of the focal and the reference group, the sample size, the proportion of DIF items and the length of the test.