The effects of interviewer,respondent and area characteristics on cooperation in panel surveys: a multilevel approach

Refusals and noncontacts generally make up the two most important components of unit nonresponse. It is important to separate noncontacts from refusals when examining survey participation, a simultaneously analysis of both components seems relevant when assessing interviewer effects. Using data from a survey of the German Youth Institute Munich, this paper presents an application of the hierarchical regression model, which offers a comprehensive way of analyzing the simultaneous effects of specific interviewer and respondent characteristics. The results provide evidence that the participation on panel surveys is subject to interviewer, respondent as well as area characteristics.     

Alternative Approaches to Multilevel Modelling of Survey Non‐Contact and Refusal

We review three alternative approaches to modelling survey non‐contact and refusal: multinomial, sequential, and sample selection (bivariate probit) models. We then propose a multilevel extension of the sample selection model to allow for both interviewer effects and dependency between non‐contact and refusal rates at the household and interviewer level. All methods are applied and compared in an analysis of household non‐response in the United Kingdom, using a data set with unusually rich information on both respondents and non‐respondents from six major surveys. After controlling for household characteristics, there is little evidence of residual correlation between the unobserved characteristics affecting non‐contact and refusal propensities at either the household or the interviewer level. We also find that the estimated coefficients of the multinomial and sequential models are surprisingly similar, which further investigation via a simulation study suggests is due to non‐contact and refusal having largely different predictors.     

Unit nonresponse errors in income surveys: a case study

A survey on the economic and social conditions of households in the city of Modena was carried out in 2002 and in 2006 (two waves) by the CAPP (Centre for Analyses of Public Policies). In the first wave of 2002, each designated sampling unit (i.e., the family) had three units as reserves. If the first refused to be interviewed, the interviewer contacted the three reserves, one after the other, until obtaining either one respondent or four non-participant units. At the end of the survey four categories of units were distinguished: interviewees, refusals, noncontacts, and unused reserves. All units were matched with their corresponding record in the databases of the Ministry of Finance of 2002 and the Census of 2001. The resulting data set permitted the analysis of unit or total nonresponses. The distribution of fiscal income showed different shapes for the four categories, implying a selective participation of the families. The interviewees yielded a positive bias of about 600?, holding constant other factors. The selection of the significant factors affecting nonresponse was performed via backward elimination in a logit model and with the lasso method. Participation increased as fiscal income and age increased and by education level (secondary school and university degree), while it decreased among entrepreneurs, independent workers, managers, and medium-to-low skilled workers.     

Measuring Interviewer Performance in Telephone Surveys

In this research paper, the definition of interviewer performance is restricted to mean the capacity of an interviewer to convince a respondent to cooperate. Furthermore, it is restricted to performance in telephone surveys. To measure this aspect of performance, researchers have until now used the cooperation rate, calculated as the number of completed interviews divided by the sum of completes plus refusals, generally retaining only the first contacts made to a given phone number in order to avoid contaminating the measure with the performance of a previous interviewer. In this paper, we argue that this measure is biased because it does not allow for the measurement of performance when interviewers, usually the best ones, work on refusal conversion or on appointments. We propose a new measure of the net contribution of interviewers to the overall cooperation rate – the Net Contribution to Performance Index (NCPI) – that measures interviewer performance irrespective of the task they perform. The properties of the index are compared with the cooperation rate at first contact using a number of criteria: reliability, variability and relationship with the overall response rate, and internal and external validity. Appendix A explains how to calculate the new index.     

Understanding applicant behavior in employment interviews: A theoretical model of interviewee performance

The purpose of this article is to present a theoretical model of interviewee performance in selection interviews. Our model positions the construct of interviewee performance as a central mediating variable between candidate attributes and interviewer ratings. The model includes six sets of factors that may influence interviewee performance, interviewer ratings, or both (e.g., interviewer–interviewee dynamics). This model promotes a fundamental shift in the way we think about employment interviews, from a focus on interviewer ratings to a focus on interviewee performance. Factors like culture and interview specific self-efficacy, while receiving little attention in current literature, take on greater significance when viewed through the lens of interviewee performance. A number of avenues for future research are developed and presented, which we hope will encourage future research in this area.     

Selective Response to Questions on Delinquency*

In the German General Survey 2000 (ALLBUS), the so-called ‘Sealed Envelope Technique’ (SET), was utilized to obtain data on an individuals’ self-admitted delinquency. The focus of this article is to discover, particularly, the reason respondents refused to fill in this confidential questionnaire in spite of the guaranteed anonymity. From a theoretical perspective of subjective expected utility, the assumption is that respondents are interested in maximizing benefits and avoiding social costs in the interview situation. Consequently, responses provided are optimal realizations of the respondents’ interest. Furthermore, the respondents’ intellectual capacity in understanding the questions, the SET applied, the interviewer characteristics, and aspects of the interview situation, were presumably responsible for refusals on sensitive questions. The ALLBUS 2000 data confirm these hypotheses. The selectivity of self-reported delinquency on matters concerning fare avoidance and tax evasion also resulted in biased model estimators of determinants regarding anticipated future delinquency. Mail survey is one supported view on improving data quality in self-admitted acts of delinquency. However, before firm conclusions can be drawn, more empirical data is needed on the processes and mechanisms involved in a respondents refusal to answer questions on delinquency. * An empirical assessment on the effectiveness of the ‘Sealed Envelope Technique’ for self-admitted delinquency through the utilization of the German General Social Survey 2000 data.     

Certifying interviewers: the role of testing interviewers to improve data quality

The complexity of conducting personal interviews often requires both classroom training prior to fielding a survey and the certification of interviewers in the field. In an attempt to create a tool that would enable data collection agencies to tailor the training process and more effectively link the classroom to the field, the Census Bureau administered a certification exam to the interviewers of a new, more conversational style of interviewing in the redesign of the Survey of Income and Program Participation-Event History Calendar (SIPP-EHC). This research applied multilevel modeling to the 2010 and 2011 SIPP-EHC field test data to observe the effects of the variation in certification test score on measures of interviewer productivity—interview length and person non-response. From 2010 to 2011, short answer response questions replaced half of the multiple-choice response questions, finding short answer responses are better predictors of both interview length and person non-response than multiple-choice responses. These changes in the exam demonstrated a significant reduction in the interviewer effect seen on both outcomes.     

Explaining interviewer effects and respondent behavior: Theoretical models and empirical analysis

The behavior of respondents in interview situations has been dealt on the one hand with respect to many empirical studies and on the other hand in connection with different theoretical approaches (Hyman 1954; Cannell & Kahn 1968). In this paper the most relevant theoretical explanations are discussed and systematized from the point of view of the theory of reasoned action (Ajzen & Fishbein 1980). Whereas this theory program has been used in many substantive fields, it has rarely been applied to the problem of interviewer effects and response sets. In this approach one assumes that the actors in interview situations decide according to cost-benefit calculations. The theory of reasoned action is viewed as an operationalized theory, discussed in more detail and formalized via structural equation models. These models are empirically tested with the data of a survey specifically designed to perform such a method study. The reported contact rates of German with foreigners is the dependent variable under study. First, a model without interviewer variables is tested to explain respondent behavior in terms of norms, attitudes and some other determinants. Then the status and the age of interviewers are introduced as situational determinants of the respondents' behavior. For subgroup analyses the respondents are divided into three groups varying in the amount of the need for social approval. The models are tested according to two subgroups (low and high need for social approval) with the technique of multiple group comparison in LISREL (Jöreskog/Sörbom 1988). All models and results are interpreted in terms of the theory of reasoned action. At the end some conclusions for modelling interviewer effects and respondent behavior are discussed.     

Two new models for survey sampling with sensitive characteristic: design and analysis

Sensitive topics or highly personal questions are often being asked in medical, psychological and sociological surveys. This paper proposes two new models (namely, the triangular and crosswise models) for survey sampling with the sensitive characteristics. We derive the maximum likelihood estimates (MLEs) and large-sample confidence intervals for the proportion of persons with sensitive characteristic. The modified MLEs and their asymptotic properties are developed. Under certain optimality criteria, the designs for the cooperative parameter are provided and the sample size formulas are given. We compare the efficiency of the two models based on the variance criterion. The proposed models have four advantages: neither model requires randomizing device, the models are easy to be implemented for both interviewer and interviewee, the interviewee does not face any sensitive questions, and both models can be applied to both face-to-face personal interviews and mail questionnaires.     

Workplace Religious Displays and Perceptions of Organization Attractiveness

Workplace religious expression has become an intensely debated topic across news outlets and social media. However research on what constitutes acceptable vs unacceptable workplace religious display is sparse. At a time when EEOC claims involving religion are on the rise there is a clear need for study in this area. In this study participants in two samples read 27 scenarios where an interviewer engaged in a Christian religious display during a job interview. We used Christian religious displays for their ease of recognition in an American sample. Participants rated each workplace religious display in terms of likelihood of occurrence and organization attractiveness. In both samples organization attractiveness ratings were more negative than expected in a predominantly Christian sample signifying that while individuals may value their ability to express their religion they may not appreciate such displays from those who represent an organization. Verbal and physical religious displays received more negative ratings compared to scenarios that spoke to shared experiences such as displaying pictures of one’s children in a religious ceremony. Application in organizations and HR implications are discussed.     

Interviewer-related measurement error in attitudinal research: a nonexperimental study

Prior research efforts have attempted to identify the extent to which interviewers bias survey data. Interviewer characteristics such as age, sex, race and social class have been widely studied, as have the effects of certain interviewer attitudes. These previous works, however, have examined the effect of only one or two of these variables upon responses. The current research incorporates interviewer demographic variables, social distance measures, and interviewer attitudinal indicators into the analysis to ascertain their relative impact upon respondent attitudes. Findings include: (1) interviewer demographic characteristics and social distance measures have no significant effects upon responses; and (2) interviewer attitudes exert a statistically significant but small influence upon respondent attitudes.     

Refusals to Deal, Price Discrimination, and Independent Service Organizations

A number of recent Canadian and U.S. antitrust cases have involved allegations that manufacturers of durable products have refused to supply parts to independent service organizations, apparently to monopolize the market for repairs of their products. This paper provides a theory of these strategies and considers the welfare implications of judicial orders to supply. The refusals here are seen as necessary to protect manufacturers' program of price discrimination: Expensive repairs represent a way to select high-intensity, high-value users and charge them more. In addition to the usual ambiguity associated with the welfare effects of prohibitions of price discrimination, forcing competition in repairs can have the further damaging effect of reducing social welfare by inducing manufacturers to lower product quality.     

Question-Answer Sequences in Survey-Interviews

Interaction analysis was used to analyze a total of 14,265 question–answer sequences of (Q-A Sequences) 80 questions that originated from two face-to-face and three telephone surveys. The analysis was directed towards the causes and effects of particular interactional problems. Our results showed that problematic respondent behavior is affected by the questionnaire design, whereas inadequate interviewer behavior is affected by respondent behavior, rather than directly by the questionnaire design. Two surveys used questions for which validating information was available. It appeared that the occurrence of such irregularities of interviewer and respondent behavior was related to the validity of the eventual responses. Explanations for the occurrence of problematic respondent behavior were proposed, concerning both cognitive and conversational factors, related to the wording of questions and response alternatives. Part of this paper was presented at the International Conference on Questionnaire Development, Evaluation and Testing Methods, 14–17 November 2002, Charleston, South Carolina.     

The effects of technical and social conditions on workplace trust

This paper investigates the extent to which the technical and social contexts of organizations independently affect levels of workplace trust. We argue that, in an organizational context, trust is not just a relationship between an individual subject (the truster) and an object (the trustee) but is subject to effects from the conditions of the work relationship itself. We describe the organizational context as comprising both a technical system of production (where work gets done through the specification of tasks) and a social system of work (where problems of effort, compliance, conformity and motivation are managed). We analyse the relationship between trust and these two aspects of workplace context (technical and social systems). We also operationalize this in terms of differences between industries, occupational composition and human resource management practices. The model is tested using data drawn from the 1995 Australian Workplace Industrial Relations Survey. The results confirm that differences in industry, occupational composition and HRM practices all impact on levels of workplace trust. We review these results in terms of their implications for future research into the problem of analysing variation in trust at both the workplace and individual levels.     

The Development and Empirical Testing of Extended Input-Output Price Models

The focus in this paper is on the input-output price model as initially developed by Leontief nearly 60 years ago. A number of methodological refinements are proposed, including the formulation, for the first time, of an extended price model, with a disaggregated household sector. This model is presented in both static and dynamic versions. The effects of these refinements are investigated empirically by reference to an example of policy analysis. This relates to a hypothetical proposal to remove energy subsidies at the national level in Iran. The paper reports on the different results produced by each form of price model and also provides evidence on the sensitivity of individual parameter values. The paper concludes by considering the feasibility of constructing more comprehensive versions of the price model and identifying those elements of the model for which data are likely to be more difficult to obtain at national and regional levels.     

Politics in wage setting: does government colour matter?

This paper studies the relationship between wage formation and the political colour of the government in an economy with a centralized wage bargaining system. Ideological, organizational and personal ties between the central trade union and the social democratic political party suggest that the trade union may behave significantly different in wage negotiations under a social democratic than under a conservative government. Using time series data for Norway, we estimate that changing from a conservative to a social democratic central government significantly reduces manufacturing wages and makes wages more responsive to unemployment. This result is consistent with a wage bargaining model augmented by political preferences of the union leaders and suggests that the effect of bargaining coordination depends on the political colour of the government. The estimated effects are both robust with respect to model specification and stable over time. We are grateful to Fredrik Wulfsberg, participants at seminars in Trondheim and Oslo, and an anonymous referee for valuable comments.     

Interviewers,Interviewer Continuity,and Panel Survey Nonresponse

It is widely known that in practice, different interviewers have different response rates, though there has been no systematic examination of whether this is because of differences among interviewers or differences among those areas allocated to the interviewers (area effects), or both. Furthermore, the conventional wisdom in survey research suggests that it is advisable to have the same interviewers return to the same respondents in order to maintain good response rates in longitudinal surveys, though once again there has been very little documented experimental research to support this. This paper makes use of the interpenetrated sample design experiment in Wave 2 of the British Household Panel Study (BHPS) (i) to explore the effects of interviewers' background characteristics and years of experience on response rates, (ii) to identify and estimate the differential effects of interviewers on response rates and compare the magnitudes of area and interviewer effects, and (iii) to investigate the impact of interviewer continuity. The analysis is facilitated by the use of cross-classified multilevel modelling. The paper also looks at the issue of interviewer continuity qualitatively, through the impressions of the interviewers themselves.     

The Role of Ethnicity and Language in Contingent Valuation Analysis

A bstract .   In order to satisfy legal requirements, many federal agencies must assess the potential effects of their policies on the public. This is often done through surveys, but frequently those surveys are only administered in English. This paper tests whether there are differences in survey response rates, refusals to pay, and willingness to pay (WTP) across different ethnicities and language for forest fire reduction in the State of California. The ethnicities studied were Caucasian, African American, and Hispanic (half in Spanish, half in English).
There was a statistical difference in survey response rates across all ethnicities, and no statistical difference among ethnicities for reasons of refusing to pay. The influence of ethnicity and language was tested using a logit model with ethnicity intercepts and bid slope interaction terms. The Hispanic-Spanish intercept shifter and the Hispanic-English dollar bid amount interaction terms were statistically significant and positive. There was a significant difference in the logit willingness to pay coefficients between Hispanics surveyed in Spanish with each of the other ethnicities. The annual willingness to pay of Hispanics taking the survey in Spanish was twice that of Caucasians, but no statistical difference in mean and median WTP between these two groups was found, due to large confidence intervals around each estimate. Nonetheless, the WTP of both Hispanics and Caucasians for the forest thinning program is substantial, and statistically different from zero, suggesting there may be broad support for this program in California.     

The Importance of Experimental Control in Testing the Impact of Interviewer Continuity on Panel Survey Nonresponse

The conventional wisdom in survey research suggests that it is advisable to have the same interviewers return to the same respondents in order to maintain good response rates in longitudinal surveys. There has been, however, very little documented experimental research to support this. Work conducted by Campanelli and O'Muircheartaigh (1999) using a subsample of the British Household Panel Study (BHPS) at Wave 2 with experimental control of the allocation of respondents to interviewers showed no evidence of a positive continuity effect on nonresponse; more extensive analysis by Laurie et al. (1999) of the full BHPS sample using Waves 2 through 4 presents contradictory results. This paper extends the earlier analysis and shows that these differences in findings are due to the lack of experimental control for the inferences from the full BHPS sample in the Laurie et al. (1999) report rather than the shorter time frame considered in Campanelli and O'Muircheartaigh (1999). This paper also considers variation in interviewer continuity effects across areas through the use of multilevel statistical models.