An econometric method of correcting for unit nonresponse bias in surveys

Past approaches to correcting for unit nonresponse in sample surveys by re-weighting the data assume that the problem is ignorable within arbitrary subgroups of the population. Theory and evidence suggest that this assumption is unlikely to hold, and that household characteristics such as income systematically affect survey compliance. We show that this leaves a bias in the re-weighted data and we propose a method of correcting for this bias. The geographic structure of nonresponse rates allows us to identify a micro compliance function, which is then used to re-weight the unit-record data. An example is given for the US Current Population Surveys, 1998–2004. We find, and correct for, a strong household income effect on response probabilities.     

ESTIMATING, EXPLAINING, AND TARGETING POVERTY IN BELIZE: IMPLICATIONS FOR POLICY

This paper focuses on estimating, explaining, and targeting poverty in Belize. The analysis is based on household-level data derived from a national income and expenditure survey conducted in 1990. The paper estimates that the incidence of poverty was about 23 percent at that time nationwide, although there were significant differences in the incidence of poverty across rural and urban areas and across districts and subdistricts of the country. The analysis shows that the household head's age and education level and the household's size, number of children, and location appear to be significant explanators of the incidence of poverty; and, on the other hand, the investigation demonstrates that the head's gender, migration status, worker type, and sector of employment do not seem to explain, to any significant degree, the existence of poverty. The paper also illustrates the importance of sound targeting procedures in keeping poverty reduction costs low as a greater number of beneficiaries are included in such anti-poverty programs. In addition, the analysis shows the clear tradeoff between undercoverage and leakage errors in program implementation. Finally, the paper suggests that poverty program geographic targeting errors would probably not be reduced substantially if subdistricts were used as targets instead of districts in Belize.     

Accuracy of self-reports to the burdensome question: survey response and nonresponse error trade-offs

Survey self-reports to questions involving respondent burden are examined for: (1) overall accuracy, (2) direction of bias, and (3) influence on relationships. Self-reports by respondents to questions on an area of household finances are compared to error-free supplier records. The evidence shows self-reports in this case to be accurate at the overall, or aggregate, level, but inaccurate at the individual, or disaggregate, level. The trade-offs inherent in the nonresponse occurring as a consequence of attempting to acquire verified data versus response bias resulting from a reliance on self-reports from surveys are discussed. It is suggested that attempts by researchers to acquire error-free, verified information in surveys will lead in most cases to unacceptably high levels of item nonresponse. An alternative, algebraic procedure for adjusting for disaggregate level response bias in self-reports on the basis of a subsample of verified responses is provided.     

Is there a 'safe area' where the nonresponse rate has only a modest effect on bias despite non-ignorable nonresponse?

Rising nonresponse rates in social surveys make the issue of nonresponse bias contentious. There are conflicting messages about the importance of high response rates and the hazards of low rates. Some articles (e.g. Groves and Peytcheva, 2008) suggest that the response rate is in general not a good predictor of survey quality. Equally, it is well known that nonresponse may induce bias and increase data collection costs. We go back in the history of the literature of nonresponse and suggest a possible reason to the notion that even a rather small nonresponse rate makes the quality of a survey debatable. We also explore the relationship between nonresponse rate and bias, assuming non-ignorable nonresponse and focusing on estimates of totals or means. We show that there is a ‘safe area’ enclosed by the response rate on the one hand and the correlation between the response propensity and the study variable on the other hand; in this area, (1) the response rate does not greatly affect the nonresponse bias, and (2) the nonresponse bias is small.     

Nonresponse and Recall Errors in a Study of Absence because of Illness: An Analysis of Their Effects on Distributions and Relationships

Using administrative data as validating standard, we studied the combined effects of two sources of survey error – nonresponse and recall errors – on distributional and substantive bias in a mail survey of absence because of illness among the employees of a Dutch road building company (response rate 77%). No distributional bias was found in five socio-demographic variables (sex, age, years of service, function, and district), but both nonresponse bias and recall bias occurred in our central dependent variables: frequency and duration of absence because of illness. Nonrespondents were on sick leave more frequently and longer than respondents. Furthermore, the self-reports of absence because of illness of our respondents proved to be rather inaccurate. Underreporting of frequency and duration of sick leave was more common than overreporting. Therefore, both sources of error had a cumulative effect.While nonresponse did not result in biased relationships, recall errors had clearly biasing consequences: seven out of 30 correlation coefficients analyzed were too biased to produce valid outcomes; another six were substantially biased. Multiple regression used for predicting recent absence because of illness among our respondents also led to different outcomes depending on the choice of data source (administration or questionnaire) for our absence variables.     

Nonresponse Bias and Stimulus Effects in the Dutch National Election Study

Bias is a much-debated issue in survey research. Answer effects (respondents claim to have behaved differently than they did in reality), nonresponse bias (nonrespondents differ on important variables from the respondents) and stimulus effects (by participating in a previous wave of a study, respondents change their behavior or attitude) can seriously distort the results of survey research. By using data from the 1998 Dutch National Election Study the authors show that the results of election research can indeed be affected by bias. Not only are significant effects found in the distribution of political attitude and voting behavior variables as a result of both nonresponse bias and stimulus effects, it is also shown that relations between variables change as a result of bias.     

Reducing Missing Data in Surveys: An Overview of Methods

Although item nonresponse can never be totally prevented, it can be considerably reduced, and thereby provide the researcher with not only more useable data, but also with helpful auxiliary information for a better imputation and adjustment. To achieve this an optimal data collection design is necessary. The optimization of the questionnaire and survey design are the main tools a researcher has to reduce the number of missing data in any such survey. In this contribution a concise typology of missing data patterns and their sources of origin are presented. Based on this typology, the mechanisms responsible for missing data are identified, followed by a discussion on how item nonresponse can be prevented.     

The components of landline telephone survey coverage bias. The relative importance of no-phone and mobile-only populations

The continuously growing mobile-only population raises concerns regarding the representativeness of traditional landline telephone surveys. At this time, the mobile-only population differs significantly from general population, which leads to coverage bias when using fixed-line samples only for telephone surveys. However, in many European countries the mobile-only population is not the only source of coverage bias in telephone surveys. In addition, we have to consider coverage biases caused by considerable proportions of citizens without any telephone service. Since these two groups differ from the general population with respect to differential socio-demographic categories, in our view, the negative effects of mobile-only coverage error in traditional landline telephone surveys might in fact compensate—in part—for coverage bias caused by the no-phone population. To test this hypothesis of compensating coverage biases we calculated relative coverage biases caused by the mobile-only population and relative coverage biases caused by the no-phone population in 30 European countries for two socio-demographic variables in two points in time. Results are presented for four groups of countries that differ with respect to no-phone and mobile-only rates. Results suggest that—in general—mobile-only biases and no-phone biases do not compensate to a great extent, and thus the alarming mobile-only biases cannot be neglected when using telephone surveys in the estimation of population parameters. Nevertheless, there are several countries where the bias caused by the mobile-only population is far bigger than the joint bias caused by the mobile-only population and the no-phone population. This finding suggests that biases caused by the recent mobile-only population would be even more severe if the no-phone population did not exist.     

Post‐Stratification Weighting in Organizational Surveys: A Cross‐Disciplinary Tutorial

Post‐stratification weighting is a technique used in public opinion polling to minimize discrepancies between population parameters and realized sample characteristics. The current paper provides a weighting tutorial to organizational surveyors who may otherwise be unfamiliar with the rationale behind the practice as well as “when and how to do” such weighting. The primary reasons to weight include: [1] reducing the effect of frame, sampling, and nonresponse bias in point estimates, and, relatedly, (2) correcting for aggregation error resulting from over‐ and underrepresentation of constituent groups. We briefly compare and contrast traditions within public opinion and organizational polling contexts and present a hybrid taxonomy of sampling procedures that organizational surveyors may find useful in situating their survey efforts within a methodological framework. Next, we extend the existing HRM literature focused on survey nonresponse to a broader lens concerned with population misrepresentation. It is from this broadened methodological framework that we introduce the practice of weighting as a remedial strategy for misrepresentation. We then provide sample weighting algorithms and standard error corrections that can be applied to organizational survey data and make our data and procedures available to individuals who may wish to use our examples as they learn “how to weight.” © 2018 Wiley ­Periodicals, Inc.     

Estimation Techniques for Ordinal Data in Multiple Frame Surveys with Complex Sampling Designs

Surveys usually include questions where individuals must select one in a series of possible options that can be sorted. On the other hand, multiple frame surveys are becoming a widely used method to decrease bias due to undercoverage of the target population. In this work, we propose statistical techniques for handling ordinal data coming from a multiple frame survey using complex sampling designs and auxiliary information. Our aim is to estimate proportions when the variable of interest has ordinal outcomes. Two estimators are constructed following model‐assisted generalised regression and model calibration techniques. Theoretical properties are investigated for these estimators. Simulation studies with different sampling procedures are considered to evaluate the performance of the proposed estimators in finite size samples. An application to a real survey on opinions towards immigration is also included.     

Trust in government and its effect on preferences for income redistribution and perceived tax burden

This paper explores how trust in government shared by neighbors is associated with individual preferences for income redistribution and individual perceptions regarding income tax burden. Using individual-level data from Japan, a nation with shared homogenous racial and cultural backgrounds, this paper controls for unobservable heterogeneity among Japan’s population, thus reducing estimation bias. Three measures for trust in government are used: “trust in ministries and government agencies”, “trust in diet members”, and “trust in members of municipal councils”. After controlling for individual characteristics, the key findings are: (1) people are more likely to express preferences for income redistribution when trust in government in their residential area is high; (2) people are more likely to perceive their tax burden as low when trust in government in their residential area is high; and (3) when the sample is divided into people with above average income and those with below average income, these results are only clearly observed for people with above average income and not those with below average income.     

The use of proxy variables in housing price analysis

This paper presents a reexamination of data used by Berry to study housing prices in Chicago. The detailed data on 275 single-family houses are used to test the proposition that the tax assessment on improvements is a good proxy for the attributes of the houses. It is shown that the test used by Berry is irrelevant for the question of omitted variables bias, and the correct test is presented. It is concluded that the proxy corrects for a bias in the coefficient of percent black population, but increases the negative bias in the coefficient of percent Latino population.     

On selectivity of nonresponse in discrete-time multi-wave panel studies

The current paper deals with the issue of the detection of selective nonresponse in discrete-time, multi-wave panel studies. If groups in a population differ with respect to the chances that they will be (and remain) in a longitudinal sample, we speak of selective nonresponse. Ultimately, selective nonresponse may lead to a sample that is very different from the target population. We discuss ways to detect and quantify the amount of selectiveness by means of discrete-time Markov models. Then we proceed by addressing how a researcher may gain understanding of how to solve the problems caused by selective nonresponse, and the degree to which these solutions will be effective, by means of data on the nonresponse during a three-wave panel study involving 2800 young Dutch adults.     

Nonresponse in panel data: The impact on estimates of a life cycle consumption function

If missing observations in a panel data set are not missing at random, many widely applied estimators may be inconsistent. In this paper we examine empirically several ways to reveal the nature and severity of the selectivity problem due to nonresponse, as well as a number of methods to estimate the resulting models. Using a life cycle consumption function and data from the Expenditure Index Panel from the Netherlands, we discuss simple procedures that can be used to assess whether observations are missing at random, and we consider more complicated estimation procedures that can be used to obtain consistent or efficient estimates in case of selectivity of attrition bias. Finally, some attention is paid to the differences in identification, consistency, and efficiency between inferences from a single wave of the panel, a balanced sub-panel, and an unbalanced panel.     

A comparison of nonresponse in mail,telephone, and face-to-face surveys

This article reports a meta-analysis of 45 studies that explicitly compare the response obtained using a mail, telephone or face-to-face survey. The data analysis uses a generalized hierarchical linear model. Sampling procedure (e.g., local convenience sample, random general sample), saliency of topic, and research organization (university, government versus market research) had an effect on the response. On the average, the face-to-face condition achieved the highest completion rate (70.3%), the telephone survey the next highest (67.2%), and the mail survey the lowest (61.3%). There is a significant interaction with the year of publication: The response to face-to-face and telephone surveys is going down in the period covered by this analysis (1947 to 1992), but the response to mail surveys is going up slightly. We attribute this to the large amount of research on nonresponse problems with mail surveys, and recommend more research and development in this direction for face-to-face and telephone methods.     

Optimal estimates and optimal predictors of finite population characteristics in the presence of nonresponse

Summary When samples from a finite population are studied, for instance by interview, there will usually be some units from which no response is obtained. In this paper optimal predictors of finite population characteristics, when nonresponse is present, are studied. The predictors are studied under simple regression superpopulation models. The optimal predictors are connected to the classical weighted sample estimates which are shown to be maximum likelihood estimates, provided the probability function is fully described by the sampling design. The predictors are compared with respects to their efficiencies for some simple models and a possible explanation to the fact that the poststratification estimate which compensate for nonresponse does no better than the simple estimate, is pointed out.     

北京市市域内人户分离观察

通过分析北京市城市内部人户分离现象发现,20%以上的户籍人口处于人户分离状态,其中以中青年为主;50%的人户分离人口从城市功能核心区到城市功能拓展区居住,且将长期分离下去。认为人户分离是社会经济发展和现行管理制度不相适应的结果,只有制度改革创新才是治本之策。     

Assessing nonresponse bias in activist surveys

How confident can we be that the comparatively low response rates associated with mail surveys of groups of political activists, such as participants of a demonstration, does not hide a substantial nonresponse bias? The paper compares the results of a face-to-face survey of 2003 anti-Iraq war demonstrators in Glasgow, achieving a near perfect response rate, with the data derived from a mail survey handed out to demonstrators eliciting valid responses from 37% of marchers. The comparison shows that better educated, older, female demonstrators were more likely to return the mail questionnaire. Also demonstrators having born a higher ‘cost’ of travelling to the demonstration are more likely to respond. There was no evidence that political interest or political orientation played an important role. However, those who had taken part in demonstrations very frequently in recent years were less likely to return the mail questionnaire. While these results provide some reassurance that even with response rates below 40%, no substantive political bias is present, researchers undertaking surveys of activists should be alerted to the need to address possible nonresponse biases in a systematic way.     

When neutrality is negative

As the percentage of the population with previous survey experience is high, it is important to study the relation between nonresponse and quality of survey experience. It was hypothesized that the pleasantness of the most recent participation in survey research would influence the willingness of persons to participate again as a respondent in surveys. Results of a survey among 981 respondents in the Netherlands indicate that significantly more respondents with a pleasant most recent survey experience had participated in the past year than respondents with an unpleasant, or indifferent most recent experience. An unpleasant or indifferent most recent participation increased the delay between participations by 50%.