Influence of equal or unequal comparison group sample sizes on the detection of differential item functioning using the Mantel–Haenszel and logistic regression techniques

In recent decades several methods have been developed for detecting differential item functioning (DIF), and many studies have aimed to identify both the conditions under which items may or may not be adequate and the factors which affect their power and Type I error. This paper describes a Monte Carlo experiment that was carried out in order to analyse the effect of reference group sample size, focal group sample size and the interaction of the two on the power and Type I error of the Mantel–Haenszel (MH) and Logistic regression (LR) procedures. The data were generated using a three-parameter logistic model, the design was fully-crossed factorial with 12 experimental conditions arising from the crossing of the two main factors, and the dependent variables were power and the rate of false positives calculated across 100 replications. The results enabled the significant factors to be identified and the two statistics to be compared. Practical recommendations are made regarding use of the procedures by psychologists interested in the development and analysis of psychological tests.     

On the impact of the tests for serial correlation upon the test of significance for the regression coefficient

Monte Carlo methods are used to investigate the relationship between the power of different pretests for autocorrelation, and the Type I error and power of the significance test for a resulting two-stage estimate of the slope parameter in a simple regression. Our results suggest it may be preferable to always transform without pretesting. Moreover we find little room for improvement in the Type I errors and power of two-stage estimators using existing pretests for autocorrelation, compared with the results obtained given perfect knowledge about when to transform (i.e., given a perfect pretest). Rather, researchers should seek better estimators of the transformation parameter itself.     

Evaluating estimation methods for ordinal data in structural equation modeling

This study examined the performance of two alternative estimation approaches in structural equation modeling for ordinal data under different levels of model misspecification, score skewness, sample size, and model size. Both approaches involve analyzing a polychoric correlation matrix as well as adjusting standard error estimates and model chi-squared, but one estimates model parameters with maximum likelihood and the other with robust weighted least-squared. Relative bias in parameter estimates and standard error estimates, Type I error rate, and empirical power of the model test, where appropriate, were evaluated through Monte Carlo simulations. These alternative approaches generally provided unbiased parameter estimates when the model was correctly specified. They also provided unbiased standard error estimates and adequate Type I error control in general unless sample size was small and the measured variables were moderately skewed. Differences between the methods in convergence problems and the evaluation criteria, especially under small sample and skewed variable conditions, were discussed.     

Controlling for ability using test scores

This paper proposes a semiparametric method to control for ability using standardized test scores, or other item response assessments, in a regression model. The proposed method is based on a model in which the parameter of interest is invariant to monotonic transformations of ability. I show that the estimator is consistent as both the number of observations and the number of items on the test grow to infinity. I also derive conditions under which this estimator is root‐n consistent and asymptotically normal. The proposed method is easy to implement, does not impose a parametric item response model, and does not require item‐level data. I demonstrate the finite‐sample performance in a Monte Carlo study and implement the procedure for a wage regression using data from the NLSY1979.     

Homogeneity tests in meta-analysis: a Monte Carlo comparison of statistical power and Type I error

The statistical power and Type I error rate of several homogeneity tests, usually applied in meta-analysis, are compared using Monte Carlo simulation: (1) The chi-square test applied to standardized mean differences, correlation coefficients, and Fisher's r-to-Z transformations, and (2) S&H-75 (and 90 percent) procedure applied to standardized mean differences and correlation coefficients. Chi-square tests adjusted correctly Type I error rates to the nominal significance level while the S&H procedures showed higher rates; consequently, the S&H procedures presented greater statistical power. In all conditions, the statistical power was very low, particularly when the sample had few studies, small sample sizes, and presented short differences between the parametric effect sizes. Finally, the criteria for selecting homogeneity tests are discussed.     

On Tests for a Normal Mean with Known Coefficient of Variation

Hinkley (1977) derived two tests for testing the mean of a normal distribution with known coefficient of variation (c.v.) for right alternatives. They are the locally most powerful (LMP) and the conditional tests based on the ancillary statistic for μ. In this paper, the likelihood ratio (LR) and Wald tests are derived for the one‐ and two‐sided alternatives, as well as the two‐sided version of the LMP test. The performances of these tests are compared with those of the classical t, sign and Wilcoxon signed rank tests. The latter three tests do not use the information on c.v. Normal approximation is used to approximate the null distribution of the test statistics except for the t test. Simulation results indicate that all the tests maintain the type‐I error rates, that is, the attained level is close to the nominal level of significance of the tests. The power functions of the tests are estimated through simulation. The power comparison indicates that for one‐sided alternatives the LMP test is the best test whereas for the two‐sided alternatives the LR or the Wald test is the best test. The t, sign and Wilcoxon signed rank tests have lower power than the LMP, LR and Wald tests at various alternative values of μ. The power difference is quite large in several simulation configurations. Further, it is observed that the t, sign and Wilcoxon signed rank tests have considerably lower power even for the alternatives which are far away from the null hypothesis when the c.v. is large. To study the sensitivity of the tests for the violation of the normality assumption, the type I error rates are estimated on the observations of lognormal, gamma and uniform distributions. The newly derived tests maintain the type I error rates for moderate values of c.v.     

Preliminary test Liu estimators based on the conflicting W,LR and LM tests in a regression model with multivariate Student-<Emphasis Type="Italic">t</Emphasis> error

In this paper, the problem of estimation of the regression coefficients in a multiple regression model with multivariate Student-t error is considered under the multicollinearity situation when it is suspected that the regression coefficients may be restricted to a linear manifold. The preliminary test Liu estimators (PTLE) based on the Wald, Likelihood ratio (LR) and Lagrangian multiplier (LM) tests are given. The bias and mean square error (MSE) of the proposed estimators are derived and conditions of superiority of these estimators are provided. In particular, we show that in the neighborhood of the null hypothesis, the PTLE based on the LM test has the best performance followed by the estimators based on LR and W tests, while the situation is reversed when the parameter moves away from the manifold of the restriction. Furthermore, the optimum choice of the level of significance is also discussed.     

Type I error and test power of different tests for testing interaction effects in factorial experiments

A simulation study was conducted to investigate the effect of non normality and unequal variances on Type I error rates and test power of the classical factorial anova F‐test and different alternatives, namely rank transformation procedure (FR), winsorized mean (FW), modified mean (FM) and permutation test (FP) for testing interaction effects. Simulation results showed that as long as no significant deviation from normality and homogeneity of the variances exists, generally all of the tests displayed similar results. However, if there is significant deviation from the assumptions, the other tests are observed to be affected at considerably high levels except FR and FP tests. As a result, when the assumptions of factorial anova F‐test are not met or, in the case those assumptions are not tested whether met, it can be concluded that using FR and FP tests is more suitable than the classical factorial anova F‐test.     

Recession forecasting using Bayesian classification

We demonstrate the use of a Naïve Bayes model as a recession forecasting tool. The approach is closely connected with Markov-switching models and logistic regression, but also has important differences. In contrast to Markov-switching models, our Naïve Bayes model treats National Bureau of Economic Research business cycle turning points as data, rather than as hidden states to be inferred by the model. Although Naïve Bayes and logistic regression are asymptotically equivalent under certain distributional assumptions, the assumptions do not hold for business cycle data. As a result, Naïve Bayes has a larger asymptotic error rate, but converges to the error rate more quickly than logistic regression, resulting in more accurate recession forecasts with limited data. We show that Naïve Bayes outperforms competing models and the Survey of Professional Forecasters consistently for real-time recession forecasting up to 12 months in advance. These results hold under standard error measures, and also under a novel measure that varies the penalty on false signals, depending on when they occur within a cycle; for example, a false signal in the middle of an expansion is penalized more heavily than one that occurs close to a turning point.     

Testing and detecting jumps based on a discretely observed process

We propose a new nonparametric test for detecting the presence of jumps in asset prices using discretely observed data. Compared with the test in Aït-Sahalia and Jacod (2009), our new test enjoys the same asymptotic properties but has smaller variance. These results are justified both theoretically and numerically. We also propose a new procedure to locate the jumps. The jump identification problem reduces to a multiple comparison problem. We employ the false discovery rate approach to control the probability of type I error. Numerical studies further demonstrate the power of our new method.     

A score test under logistic regression models based on case–control data

We propose a score statistic to test the vector of odds ratio parameters under the logistic regression model based on case–control data. The proposed score test is based on the semiparametric profile loglikelihood function under a two-sample semiparametric model, which is equivalent to the assumed logistic regression model. The proposed score statistic has an asymptotic chi-squared distribution under the null hypothesis and an asymptotic noncentral chi-squared distribution under local alternatives to the null hypothesis. Moreover, we show that the proposed score test is asymptotically equivalent to the Wald test under the logistic regression model based on case–control data. In addition, we demonstrate that the proposed score statistic and its asymptotic distribution may be obtained by fitting the prospective logistic regression model to case–control data. We present some results on simulation and on the analysis of two real datasets.     

A simple,robust and powerful test of the trend hypothesis

In this paper we develop a simple test procedure for a linear trend which does not require knowledge of the form of serial correlation in the data, is robust to strong serial correlation, and has a standard normal limiting null distribution under either I(0)$I(0)$ or I(1)$I(1)$ shocks. In contrast to other available robust linear trend tests, our proposed test achieves the Gaussian asymptotic local power envelope in both the I(0)$I(0)$ and I(1)$I(1)$ cases. For near-I(1)$I(1)$ errors our proposed procedure is conservative and a modification for this situation is suggested. An estimator of the trend parameter, together with an associated confidence interval, which is asymptotically efficient, again regardless of whether the shocks are I(0)$I(0)$ or I(1)$I(1)$, is also provided.     

Graphical Analysis of the Two Parameters logistic Function (A Methodological Proposal)

It is proposed to study the graphical representation of the parametric space of maximumlikelihood function of two parameters logistic functions, as is used in Item Response Theory. Thisproposal is made more from a point of view of understanding rather than of discovery..     

Maximum likelihood methods for fitting the burr type XII distribution to multiply (progressively) censored life test data

Summary This paper develops mathematical and computational methodology for fitting, by the method of maximum likelihood (ML), the Burr Type XII distribution to multiply (or progressively) censored life test data. Mathematical expressions are given for approximating the asymptotic variances and covariances of the ML estimates (MLEs) of the parameters of the Burr Type XII distribution. A rigorous mathematical analysis is undertaken to investigate the existence and uniqueness of the MLEs for arbitrary sample data. The methodology of this paper is applied to progressively censored sample data arising in a life test experiment.     

Pairwise Multiple Comparisons under Violation of the Independence Assumption

Several multiple comparison procedures (MCPs)were compared for their rates of Type I error and fortheir ability to detect true pairwise differencesamong means when independence of observationsassumption were not satisfied. Monte Carlo resultsshowed that, if independence is not met, none of theprocedures maintain controlled at the chosennominal level, neither using error rate per comparisonor the error rate experimentwise. However, once thedependence of the data was corrected the Type I errorrate was maintained at the same level as when thecorrelation was zero in all the procedures, except forthe Fisher's (1935) least significant differenceprocedure (LSD) and Hayter's (1986) two-stagemodified LSD procedure (FH). At the sametime, conform the correlation increased by a smallamount the power rates also, specially, when the powerwas examined using per-pair power.     

Robust testing procedures of process locations

In many manufacturing and service industries, the quality department of the organization works continuously to ensure that the mean or location of the process is close to the target value. In order to understand the process, it is necessary to provide numerical statements of the processes that are being investigated. That is why the researcher needs to check the validity of the hypotheses that are concerned with some physical phenomena. It is usually assumed that the collected data behave well. However, sometimes the data may contain outliers. The presence of one or more outliers might seriously distort the statistical inference. Since the sample mean is very sensitive to outliers, this research will use the smooth adaptive (SA) estimator to estimate the population mean. The SA estimator will be used to construct testing procedures, called smooth adaptive test (SA test), for testing various null hypotheses. A Monte Carlo study is used to simulate the values of the probability of a Type I error and the power of the SA test. This is accomplished by constructing confidence intervals of the process mean by using the SA estimator and bootstrap methods. The SA test will be compared with other tests such as the normal test, t test and a nonparametric statistical method, namely, the Wilcoxon signed-rank test. Also, the cases with and without outliers will be considered. For the right-skewed distributions, the SA test is the best choice. When the population is a right-skewed distribution with one outlier, the SA test controls the probability of a Type I error better than other tests and is recommended.     

Bootstrap and Asymptotic Tests of Long-run Relationships in Cointegrated Systems

Hypothesis testing on cointegrating vectors based on the asymptotic distributions of the test statistics are known to suffer from severe small sample size distortion. In this paper an alternative bootstrap procedure is proposed and evaluated through a Monte Carlo experiment, finding that the Type I errors are close to the nominal signficance levels but power might be not entirely adequate. It is then shown that a combined test based on the outcomes of both the asymptotic and the bootstrap tests will have both correct size and low Type II error, therefore improving the currently available procedures.     

Preliminary test estimators of the parameters of simple linear model with measurement error

In this paper, we discuss the properties of preliminary test estimators (PTE) of the parameters of simple linear model with measurement error (ME model) when the slope of the linear model is suspected to be zero. Expressions of the bias, MSE and efficiencies are obtained under conditional as well as unconditional situations with known reliability coefficient. Conditional model results are compared to the standard model without measurement error. We also provide the unconditional model analysis in finite samples. Asymptotic theory under local alternatives is developed when the variance of measurement error or the ratio of the variance of the model error relative to the variance of the measurement error is known. Asymptotic expressions of bias and MSE of the estimators along with their efficiencies are obtained. In every case, it is shown that the measurement error tend to increase the variability of the estimators compared to the estimators without measurement error. Graphs and tables are provided to see these results and to determine optimum level of significance for minimum guaranteed efficiency. Received October 2001 RID="*" ID="*"  A. K. Md. E. Saleh is a Distinguished Research Professor and H. M. Kim is a Ph.D. candidate in the School of Mathematics and Statistics, Carleton University, Ottawa. Acknowledgment. The authors gratefully acknowledge the constructive suggestion of the referees to improve the paper. The research is supported by NSERC grant A3088.     

Estimators and hypothesis tests for a stochastic frontier function: A Monte Carlo analysis

This paper uses Monte Carlo experimentation to investigate the finite sample properties of the maximum likelihood (ML) and corrected ordinary least squares (COLS) estimators of the half-normal stochastic frontier production function. Results indicate substantial bias in both ML and COLS when the percentage contribution of inefficiency in the composed error (denoted by *) is small, and also that ML should be used in preference to COLS because of large mean square error advantages when * is greater than 50%. The performance of a number of tests of the existence of technical inefficiency is also investigated. The Wald and likelihood ratio (LR) tests are shown to have incorrect size. A one-sided LR test and a test of the significance of the third moment of the OLS residuals are suggested as alternatives, and are shown to have correct size, with the one-sided LR test having the better power of the two.The author would like to thank Bill Griffiths, George Battese, Howard Doran, Bill Greene and two anonymous referees for valuable comments. Any errors which remain are those of the author.     

Nonuniform DIF Detection using Discriminant Logistic Analysis and Multinomial Logistic Regression: A comparison for polytomous items

This study focused on the effectiveness in nonuniform polytomous item DIF detection using Discriminant Logistic Analysis (DLA) and Multinomial Logistic Regression (MLR). A computer simulation study was conducted to compare the effect of using DLA and MLR, applying either an iterative test purification procedure or non-iterative to detect nonuniform DIF. The conditions under study were: DIF effect size (0.5, 1.0 and 1.5), sample size (500 and 1000), percentage of DIF items in the test (0, 10 and 20%) and DIF type (nonuniform). The results suggest that DLA is more accurate than MLR in detecting DIF. However, the purification process only improved the correct detection rate when MLR was applied. The false positive rates for both procedures were similar. Moreover, when the test purification procedure was used, the proportion of non-DIF items that were detected as DIF decreased for both procedures, although the false positive rates were smaller for DLA than for MLR.