Erroneous detection of nonuniform DIF using the Breslow-Day test in a short test

This paper studies the Type I error rate obtained using the Breslow-Day (BD) test to detect Nonuniform Differential Item Functioning (NUDIF) in a short test when the average ability of one group is significantly higher than that of the other. The performance is compared with the logistic regression (LR) and the standard Mantel-Haenszel procedure (MH). Responses to a 20-item test were simulated without Differential Item Functioning (DIF) according to the three-parameter logistic model. The manipulated factors were sample size and item parameters. The design yielded 40 conditions that were replicated 50 times and the false positive rate at a 5% significance level obtained with the three methods was recorded for each condition. In most cases, BD performed better than LR and MH in terms of proneness to Type I error. With the BD test, the Type I error rate was similar to the nominal one when the item with the highest discrimination and difficulty parameters in the case of equally sized groups was excluded from the goodness-of-fit to the binomial distribution (number of false positives among the fifty replications of a Bernoulli variable with parameter equal to 0.05).     

Bootstrap and Asymptotic Tests of Long-run Relationships in Cointegrated Systems

Hypothesis testing on cointegrating vectors based on the asymptotic distributions of the test statistics are known to suffer from severe small sample size distortion. In this paper an alternative bootstrap procedure is proposed and evaluated through a Monte Carlo experiment, finding that the Type I errors are close to the nominal signficance levels but power might be not entirely adequate. It is then shown that a combined test based on the outcomes of both the asymptotic and the bootstrap tests will have both correct size and low Type II error, therefore improving the currently available procedures.     

Robust Small Area Estimation under Spatial Non‐stationarity

The effective use of spatial information in a regression‐based approach to small area estimation is an important practical issue. One approach to account for geographic information is by extending the linear mixed model to allow for spatially correlated random area effects. An alternative is to include the spatial information by a non‐parametric mixed models. Another option is geographic weighted regression where the model coefficients vary spatially across the geography of interest. Although these approaches are useful for estimating small area means efficiently under strict parametric assumptions, they can be sensitive to outliers. In this paper, we propose robust extensions of the geographically weighted empirical best linear unbiased predictor. In particular, we introduce robust projective and predictive estimators under spatial non‐stationarity. Mean squared error estimation is performed by two analytic approaches that account for the spatial structure in the data. Model‐based simulations show that the methodology proposed often leads to more efficient estimators. Furthermore, the analytic mean squared error estimators introduced have appealing properties in terms of stability and bias. Finally, we demonstrate in the application that the new methodology is a good choice for producing estimates for average rent prices of apartments in urban planning areas in Berlin.     

Influence of equal or unequal comparison group sample sizes on the detection of differential item functioning using the Mantel–Haenszel and logistic regression techniques

In recent decades several methods have been developed for detecting differential item functioning (DIF), and many studies have aimed to identify both the conditions under which items may or may not be adequate and the factors which affect their power and Type I error. This paper describes a Monte Carlo experiment that was carried out in order to analyse the effect of reference group sample size, focal group sample size and the interaction of the two on the power and Type I error of the Mantel–Haenszel (MH) and Logistic regression (LR) procedures. The data were generated using a three-parameter logistic model, the design was fully-crossed factorial with 12 experimental conditions arising from the crossing of the two main factors, and the dependent variables were power and the rate of false positives calculated across 100 replications. The results enabled the significant factors to be identified and the two statistics to be compared. Practical recommendations are made regarding use of the procedures by psychologists interested in the development and analysis of psychological tests.     

An inconvenient dataset: bias and inappropriate inference with the multilevel model

The multilevel model has become a staple of social research. I textually and formally explicate sample design features that, I contend, are required for unbiased estimation of macro-level multilevel model parameters and the use of tools for statistical inference, such as standard errors. After detailing the limited and conflicting guidance on sample design in the multilevel model didactic literature, illustrative nationally-representative datasets and published examples that violate the posited requirements are identified. Because the didactic literature is either silent on sample design requirements or in disagreement with the constraints posited here, two Monte Carlo simulations are conducted to clarify the issues. The results indicate that bias follows use of samples that fail to satisfy the requirements outlined; notably, the bias is poorly-behaved, such that estimates provide neither upper nor lower bounds for the population parameter. Further, hypothesis tests are unjustified. Thus, published multilevel model analyses using many workhorse datasets, including NELS, AdHealth, NLSY, GSS, PSID, and SIPP, often unwittingly convey substantive results and theoretical conclusions that lack foundation. Future research using the multilevel model should be limited to cases that satisfy the sample requirements described.     

A new procedure for monitoring the range and standard deviation of a quality characteristic

The Shewhart and the Bonferroni-adjustment R and S chart are usually applied to monitor the range and the standard deviation of a quality characteristic. These charts are used to recognize the process variability of a quality characteristic. The control limits of these charts are constructed on the assumption that the population follows approximately the normal distribution with the standard deviation parameter known or unknown. In this article, we establish two new charts based approximately on the normal distribution. The constant values needed to construct the new control limits are dependent on the sample group size (k) and the sample subgroup size (n). Additionally, the unknown standard deviation for the proposed approaches is estimated by a uniformly minimum variance unbiased estimator (UMVUE). This estimator has variance less than that of the estimator used in the Shewhart and Bonferroni approach. The proposed approaches in the case of the unknown standard deviation, give out-of-control average run length slightly less than the Shewhart approach and considerably less than the Bonferroni-adjustment approach.     

Modelling urban population density in a multi-centered city

Modelling the geographic distribution of urban population densities has been attempted in several ways. Recently a controversy emerged in the Journal of Urban Economics regarding whether or not calibrations of these models render unbiased parameter estimates. Three sources of bias were dealt with in these discussions, namely (1) model specification error, (2) the estimation procedure employed, and (3) the definition of areal unit observation size. Additional sources of bias overlooked in this controversy include the presence of multiple centers in a city, and the existence of externalities. This paper explores these additional sources, from both a conceptual and an empirical point of view.     

The Zero Lower Bound and Parameter Bias in an Estimated DSGE Model

This paper examines how and to what extent parameter estimates can be biased in a dynamic stochastic general equilibrium (DSGE) model that omits the zero lower bound (ZLB) constraint on the nominal interest rate. Our Monte Carlo experiments using a standard sticky‐price DSGE model show that no significant bias is detected in parameter estimates and that the estimated impulse response functions are quite similar to the true ones. However, as the frequency of being at the ZLB or the duration of ZLB spells increases, the parameter bias becomes larger and therefore leads to substantial differences between the estimated and true impulse responses. It is also demonstrated that the model missing the ZLB causes biased estimates of structural shocks even with the virtually unbiased parameters. Copyright © 2015 John Wiley & Sons, Ltd.     

Robust trend inference with series variance estimator and testing-optimal smoothing parameter

The paper develops a novel testing procedure for hypotheses on deterministic trends in a multivariate trend stationary model. The trends are estimated by the OLS estimator and the long run variance (LRV) matrix is estimated by a series type estimator with carefully selected basis functions. Regardless of whether the number of basis functions K is fixed or grows with the sample size, the Wald statistic converges to a standard distribution. It is shown that critical values from the fixed-K asymptotics are second-order correct under the large-K asymptotics. A new practical approach is proposed to select K that addresses the central concern of hypothesis testing: the selected smoothing parameter is testing-optimal in that it minimizes the type II error while controlling for the type I error. Simulations indicate that the new test is as accurate in size as the nonstandard test of Vogelsang and Franses (2005) and as powerful as the corresponding Wald test based on the large-K asymptotics. The new test therefore combines the advantages of the nonstandard test and the standard Wald test while avoiding their main disadvantages (power loss and size distortion, respectively).     

Estimation of bias and standard error of an improved estimator of normal mean

This paper considers an improved estimator of normal mean which is obtained by considering a feasible version of minimum mean squared error estimator. The exact expression for the bias and the mean squared error are fairly complicated and do not provide any guidelines as how to estimate the standard error of improved estimator. As is well known that any estimator without a formula for standard error has little practical utility. We therefore derive unbiased estimators for the bias and mean squared error of the improved estimator. Incidently, they turn out to be minimum variance unbiased estimators. Further, this exercise yields a simple formula for estimating the standard error. Based on the criterion of estimated standard error, the efficiency of the improved estimator with respect to the traditional unbiased estimator (i.e., sample mean) is examined numerically. The relationship with asymptotic standard error is also studied.     

The sampling distribution of forecasts from a first-order autoregression

Previous work on characterising the distribution of forecast errors in time series models by statistics such as the asymptotic mean square error has assumed that observations used in estimating parameters are statistically independent of those used to construct the forecasts themselves. This assumption is quite unrealistic in practical situations and the present paper is intended to tackle the question of how the statistical dependence between the parameter estimates and the final period observations used to generate forecasts affects the sampling distribution of the forecast errors. We concentrate on the first-order autoregression and, for this model, show that the conditional distribution of forecast errors given the final period observation is skewed towards the origin and that this skewness is accentuated in the majority of cases by the statistical dependence between the parameter estimates and the final period observation.     

On the initialization of adaptive learning in macroeconomic models

We review and evaluate methods previously adopted in the applied literature of adaptive learning in order to initialize agents’ beliefs. Previous methods are classified into three broad classes: equilibrium-related, training sample-based, and estimation-based. We conduct several simulations comparing the accuracy of the initial estimates provided by these methods and how they affect the accuracy of other estimated model parameters. We find evidence against their joint estimation with standard moment conditions: as the accuracy of estimated initials tends to deteriorate with the sample size, spillover effects also deteriorate the accuracy of the estimates of the model’s structural parameters. We show how this problem can be attenuated by penalizing the variance of estimation errors. Even so, the joint estimation of learning initials with other model parameters is still subject to severe distortions in small samples. We find that equilibrium-related and training sample-based initials are less prone to these issues. We also demonstrate the empirical relevance of our results by estimating a New Keynesian Phillips curve with learning, where we find that our estimation approach provides robustness to the initialization of learning. That allows us to conclude that under adaptive learning the degree of price stickiness is lower compared to inferences under rational expectations.     

Estimating panel data models in the presence of endogeneity and selection

We consider estimation of panel data models with sample selection when the equation of interest contains endogenous explanatory variables as well as unobserved heterogeneity. Assuming that appropriate instruments are available, we propose several tests for selection bias and two estimation procedures that correct for selection in the presence of endogenous regressors. The tests are based on the fixed effects two-stage least squares estimator, thereby permitting arbitrary correlation between unobserved heterogeneity and explanatory variables. The first correction procedure is parametric and is valid under the assumption that the errors in the selection equation are normally distributed. The second procedure estimates the model parameters semiparametrically using series estimators. In the proposed testing and correction procedures, the error terms may be heterogeneously distributed and serially dependent in both selection and primary equations. Because these methods allow for a rather flexible structure of the error variance and do not impose any nonstandard assumptions on the conditional distributions of explanatory variables, they provide a useful alternative to the existing approaches presented in the literature.     

Panel unit root tests under cross-sectional dependence

In this paper alternative approaches for testing the unit root hypothesis in panel data are considered. First, a robust version of the Dickey-Fuller t -statistic under contemporaneous correlated errors is suggested. Second, the GLS t -statistic is considered, which is based on the t -statistic of the transformed model. The asymptotic power of both tests is compared against a sequence of local alternatives. To adjust for short-run serial correlation of the errors, we propose a pre-whitening procedure that yields a test statistic with a standard normal limiting distribution as N and T tends to infinity. The test procedure is further generalized to accommodate individual specific intercepts or linear time trends. From our Monte Carlo simulations it turns out that the robust OLS t -statistic performs well with respect to size and power, whereas the GLS t -statistic may suffer from severe size distortions in small and moderate sample sizes. The tests are applied to test for a unit root in real exchange rates.     

Nearly exact sample size calculation for powerful non‐randomized tests for differences between binomial proportions

In the case of two independent samples, it turns out that among the procedures taken in consideration, BOSCHLOO'S technique of raising the nominal level in the standard conditional test as far as admissible performs best in terms of power against almost all alternatives. The computational burden entailed in exact sample size calculation is comparatively modest for both the uniformly most powerful unbiased randomized and the conservative non‐randomized version of the exact Fisher‐type test. Computing these values yields a pair of bounds enclosing the exact sample size required for the Boschloo test, and it seems reasonable to replace the exact value with the middle of the corresponding interval. Comparisons between these mid‐N estimates and the fully exact sample sizes lead to the conclusion that the extra computational effort required for obtaining the latter is mostly dispensable. This holds also true in the case of paired binary data (McNemar setting). In the latter, the level‐corrected score test turns out to be almost as powerful as the randomized uniformly most powerful unbiased test and should be preferred to the McNemar–Boschloo test. The mid‐N rule provides a fairly tight upper bound to the exact sample size for the score test for paired proportions.     

Bayesian loss given default estimation for European sovereign bonds

We develop and apply a Bayesian model for the loss rates given defaults (LGDs) of European Sovereigns. Financial institutions are in need of LGD forecasts under Pillar II of the regulatory Basel Accord and the downturn in LGD forecasts under Pillar I. Both are challenging for portfolios with a small number of observations such as sovereigns. Our approach comprises parameter risk and generates LGD forecasts under both regular and downturn conditions. With sovereign-specific rating information, we found that average LGD estimates vary between 0.46 and 0.64, while downturn estimates lay between 0.50 and 0.86.     

Estimation of the parameter of the inverse Gaussian distribution from generalised censored samples

In this paper, a new Type of censored sample referred as generalised censored sample is defined and is differentiated fromCohen's [1963] progressively censored type I and type II samples. The maximum likelihood estimate of the parameter of the inverse Gaussian distribution from generalised censored samples is obtained. An expression for the standard error of the estimate is given.     

Small‐sample bias in synthetic cohort models of labor supply

This paper investigates small‐sample biases in synthetic cohort models (repeated cross‐sectional data grouped at the cohort and year level) in the context of a female labor supply model. I use the Current Population Survey to compare estimates when group sizes are extremely large to those that arise from randomly drawing subsamples of observations from the large groups. I augment this approach with Monte Carlo analysis so as to precisely quantify biases and coverage rates. In this particular application, thousands of observations per group are required before small‐sample issues can be ignored in estimation and sampling error leads to large downward biases in the estimated income elasticity. Copyright © 2007 John Wiley & Sons, Ltd.     

Homogeneity tests in meta-analysis: a Monte Carlo comparison of statistical power and Type I error

The statistical power and Type I error rate of several homogeneity tests, usually applied in meta-analysis, are compared using Monte Carlo simulation: (1) The chi-square test applied to standardized mean differences, correlation coefficients, and Fisher's r-to-Z transformations, and (2) S&H-75 (and 90 percent) procedure applied to standardized mean differences and correlation coefficients. Chi-square tests adjusted correctly Type I error rates to the nominal significance level while the S&H procedures showed higher rates; consequently, the S&H procedures presented greater statistical power. In all conditions, the statistical power was very low, particularly when the sample had few studies, small sample sizes, and presented short differences between the parametric effect sizes. Finally, the criteria for selecting homogeneity tests are discussed.     

Assessing Variability of Complex Descriptive Statistics in Monte Carlo Studies Using Resampling Methods

Good statistical practice dictates that summaries in Monte Carlo studies should always be accompanied by standard errors. Those standard errors are easy to provide for summaries that are sample means over the replications of the Monte Carlo output: for example, bias estimates, power estimates for tests and mean squared error estimates. But often more complex summaries are of interest: medians (often displayed in boxplots), sample variances, ratios of sample variances and non‐normality measures such as skewness and kurtosis. In principle, standard errors for most of these latter summaries may be derived from the Delta Method, but that extra step is often a barrier for standard errors to be provided. Here, we highlight the simplicity of using the jackknife and bootstrap to compute these standard errors, even when the summaries are somewhat complicated. © 2014 The Authors. International Statistical Review © 2014 International Statistical Institute