Effective Repair in Survey Interviews

In standardized survey interviews with closed-ended questions it quite often occurs that respondents do something else than mentioning their choice from the set of response categories presented together with the question. In these cases their initial response is inadequate and interviewers have to initiate repair activities. By analyzing 804 question–answer sequences, stemming from parts of a survey with elderly people (N=201), we found that 322 (41%) of the initial responses were inadequate. In 74% of these 332 sequences the interviewers made an attempt at repair. Of all repair attempts, 75% were performed correctly, that is, in a nondirective way. Most of the repair attempts (86%) result in an adequate final response of the respondent. The result of the repair activities by the interviewers is that the number of sequences with an adequate response increased from 472 to 688, thus an increase of about 45%. There does not exist ‘one best way’ of repairing. The most important thing is that some repair initiative has to be taken by the interviewer and that the repair is done in a nondirective way.     

Attitudes Toward Surveys, Attitude Accessibility and the Effect on Respondents’ Susceptibility to Nonresponse

This paper analyzes whether respondents’ attitudes toward surveys explain their susceptibility to item nonresponse. In contrast to previous studies, the decision to refuse to provide income information, not to answer other questions and the probability of ‘don’t know’ responses is tested separately. Furthermore, the interviewers’ overall judgments of response willingness was included as well. Respondents with a positive and cognitively accessible attitude toward surveys were expected to adopt a cooperative orientation and were thus deemed more likely to answer difficult as well as sensitive questions. Attitudes were measured with a 16-item instrument and the response latencies were used as an indicator for attitude accessibility. We found that respondents with more favorable evaluations of surveys had lower values on all kinds of nonresponse indicators. Except for the strong effect on the prevalence of ‘don’t knows’, survey attitudes were increasingly more predictive for all other aspects of nonresponse when these attitude answers were faster and thus cognitively more accessible. This accessibility, and thus how relevant survey attitudes are for nonresponse, was found to increase with the subjects’ exposure to surveys in the past.     

Selective Response to Questions on Delinquency*

In the German General Survey 2000 (ALLBUS), the so-called ‘Sealed Envelope Technique’ (SET), was utilized to obtain data on an individuals’ self-admitted delinquency. The focus of this article is to discover, particularly, the reason respondents refused to fill in this confidential questionnaire in spite of the guaranteed anonymity. From a theoretical perspective of subjective expected utility, the assumption is that respondents are interested in maximizing benefits and avoiding social costs in the interview situation. Consequently, responses provided are optimal realizations of the respondents’ interest. Furthermore, the respondents’ intellectual capacity in understanding the questions, the SET applied, the interviewer characteristics, and aspects of the interview situation, were presumably responsible for refusals on sensitive questions. The ALLBUS 2000 data confirm these hypotheses. The selectivity of self-reported delinquency on matters concerning fare avoidance and tax evasion also resulted in biased model estimators of determinants regarding anticipated future delinquency. Mail survey is one supported view on improving data quality in self-admitted acts of delinquency. However, before firm conclusions can be drawn, more empirical data is needed on the processes and mechanisms involved in a respondents refusal to answer questions on delinquency. * An empirical assessment on the effectiveness of the ‘Sealed Envelope Technique’ for self-admitted delinquency through the utilization of the German General Social Survey 2000 data.     

Children between the age of 5 and 11: what “don’t know” answers tell us

Children are increasingly seen as competent informants on their activities, views and needs. Different abilities of children can lead to inadequate (formally or regarding the content) answers or to the misunderstanding of proceedings during an interview process. One inadequate or at least unintended (on the part of the researcher) response is a “don’t know” answer (DK). To study when and why DK answers occur, a possible cause for them in the cognitive process and their relation to age, we analysed the DK answers given in qualitative semi-structured interviews with children aged 5–11, conducted face-to-face and on the telephone. Each of 56 children was interviewed twice, once face-to-face and once via telephone. The results show that age had an effect on DK responses due to the cognitive state of the respondent and their communicative intent, but there was no difference in the number of DKs due to problems with the research instrument or due to an adequacy judgement in respect of respondent age.     

Taking ‘Don’t Knows’ as Valid Responses: A Multiple Complete Random Imputation of Missing Data

Incomplete data is a common problem of survey research. Recent work on multiple imputation techniques has increased analysts’ awareness of the biasing effects of missing data and has also provided a convenient solution. Imputation methods replace non-response with estimates of the unobserved scores. In many instances, however, non-response to a stimulus does not result from measurement problems that inhibit accurate surveying of empirical reality, but from the inapplicability of the survey question. In such cases, existing imputation techniques replace valid non-response with counterfactual estimates of a situation in which the stimulus is applicable to all respondents. This paper suggests an alternative imputation procedure for incomplete data for which no true score exists: multiple complete random imputation, which overcomes the biasing effects of missing data and allows analysts to model respondents’ valid ‘I don’t know’ answers.     

Trust and Understanding,Two Psychological Aspects of Randomized Response

This study examines two different Randomized Response methods to see whether they evoke sufficient understanding and trust, and ensure fewer evasive answers to socially sensitive questions. Two Randomized Response methods were employed by trained interviewers to study fraud: the Forced Response method, using dice, and Kuk's method, using playing cards. Respondents were selected from the files of the social security offices of three Dutch cities. A total of 334 respondents participated voluntarily in this study of two Randomized Response methods. Most respondents were known to have committed some form of fraud, and their answer on the Randomized Response question is validated with this information. The results indicate that subjects who have a better understanding of the Forced Response technique give more socially undesirable answers. The interviewer has a most important role establishing trust and understanding. Respondents who are less able to understand the instructions, e.g., have limited language abilities, develop less trust in the method.     

Sovereign debt: Reputation, seizure and reputation

The interaction between a creditor and a sovereign debtor is described as a ‘one-shot’ game with discrete actions—total or no debt-repudiation and seizure of asset holding abroad. Possible Nash equilibria where each player chooses an action as to maximize his expected payoff given his beliefs about the other player’s action and the implications of those actions on the players’ trustworthy reputation are identified. However, if reputation losses rise convexly with the players’ relative hostility, partial repudiation and seizure can be the preferred strategies. The preferred repudiation and seizure rates are analyzed under asymmetric and symmetric information about the state of the world. (JEL classification F34)     

Face to face and telephone surveys in terms of sampling representativeness: a multidimensional analysis

This paper analyzes sample structure in a survey-based research project using both face-to-face and telephone survey techniques, at a time when the telephone survey has completely ousted the face-to-face survey; particularly in electoral studies. Multiple Correspondence Analysis is applied to the data from a political poll conducted prior to regional elections held in Spain’s Autonomous Community of Galicia in which half of the questionnaires were administered by telephone and the other half via face-to-face interview. Unlike other studies—that use a partial approach based on bivariate analysis—this one examines all the sociodemographic variables simultaneously.     

Revolution, evolution, or status quo? Guidelines for efficiency measurement in health care

This paper discusses ways forward in terms of making efficiency measurement in the area of health care more useful. Options are discussed in terms of the potential introduction of guidelines for the undertaking of studies in this area, in order to make them more useful to policy makers and those involved in service delivery. The process of introducing such guidelines is discussed using the example of the development of guidelines in economic evaluation of health technologies. This presents two alternative ways forward—‘revolution’, the establishment of a panel to establish initial guidelines, or ‘evolution’—the more gradual development of such guidelines over time. The third alternative of ‘status quo’, representing the current state of play, is seen as the base case scenario. It is concluded that although we are quite a way on in terms of techniques and publications, perhaps revolution, followed by evolution is the way forward.     

Asking and answering yes/no-questions in survey interviews: a conversational approach

The authors discuss the process of asking and answering yes/no-questions in personal survey interviews from a conversational perspective. They examine the process with regard to yes/no-questions as given in the questionnaire and with regard to the yes/no-questions that interviewers may pose in the subsequent stages of a sequence, when they are probing on the respondent's initial answer or are trying to solve other problems. Hypotheses are derived from the co-operation principle and the politeness principle of conversation, and then empirically evaluated for survey interview settings. In the relatively informal stages of the answering process (subsequences) the conversations appear to go quite well according to the conversation rules, but in the beginning formal stage to a much lesser degree. In particular it has been observed that interviewers strongly prefer to ask one-sided positive yes/no-questions in the subsequences, and, secondly, that respondents usually give agreeing answers to these yes/no-questions. It is argued that this “normal” conversation strategy may seriously affect the validity of the information obtained, and, more generally, that the practical demands from the conversation rules on the interviewer's behaviour set limits on the researcher's abstract demands on that behaviour.

     

Structural stability regardless of membership turnover? The added value of blockmodelling in the analysis of network evolution

The extent to which the emergent structure of an organization remains the same, regardless of the turnover of the members, is one of the most interesting questions raised by the analyses of its evolution. This paper uses longitudinal network analysis to provide an answer to this question. Its shows the usefulness of combining both dynamics and comparative statics (here, blockmodelling) in the study of this evolution. An empirical study examines the evolution of the structure of the advice network among judges of the Commercial Court of Paris. The combination of dynamics and comparative statics answers the initial question: Radical structuralism turns out to be wrong. An answer depends on the dimensions of the structure on which observers focus. The pecking order in the advice network remains relatively stable, regardless of members’ turnover. However, social differentiation measured in terms of role relationships and division of work shows that the relational structure does not remain the same regardless of members’ turnover. Specifically, relational processes within the organization, such as collective learning, impose varying constraints on different kinds of members over time and the overall relational structure reflects these members’ changing reactions and responses to these constraints. Since the former vary, so do the latter, and so does the resulting emergent overall structure.     

Interviewers,Interviewer Continuity,and Panel Survey Nonresponse

It is widely known that in practice, different interviewers have different response rates, though there has been no systematic examination of whether this is because of differences among interviewers or differences among those areas allocated to the interviewers (area effects), or both. Furthermore, the conventional wisdom in survey research suggests that it is advisable to have the same interviewers return to the same respondents in order to maintain good response rates in longitudinal surveys, though once again there has been very little documented experimental research to support this. This paper makes use of the interpenetrated sample design experiment in Wave 2 of the British Household Panel Study (BHPS) (i) to explore the effects of interviewers' background characteristics and years of experience on response rates, (ii) to identify and estimate the differential effects of interviewers on response rates and compare the magnitudes of area and interviewer effects, and (iii) to investigate the impact of interviewer continuity. The analysis is facilitated by the use of cross-classified multilevel modelling. The paper also looks at the issue of interviewer continuity qualitatively, through the impressions of the interviewers themselves.     

Trade and product innovations as sources for productivity increases: an empirical analysis

Increases in total factor productivity (TFP) are commonly associated with technological innovations measured by the stock of R&D. Empirical evidence seems to corroborate this relationship. However, in trading countries like The Netherlands, productivity increases, even in industry, can also be the result of innovations in the way transactions are managed. These innovations reduce transaction costs and exploit the welfare gains from (further) international division of labour. Such innovations are only partly included in R&D data. Consequently there is not much attention for these ‘trade innovations’—as we label them—in policy. In an empirical analysis this paper compares the influence of trade innovations with the influence of the stock of R&D on TFP in The Netherlands. The regression results show that in this country trade innovations are as important for TFP as technological innovations which directly affect the efficiency of production, which we label ‘product innovations’.      

University training for entrepreneurial competencies: Its impact on intention of venture creation

This study tests the effect of entrepreneurship education programmes on the entrepreneurial competencies and intention of university students in order to confirm (or disconfirm) conventional wisdom that entrepreneurial education increases the intention to start a business. We address the following research question: Do entrepreneurship education programmes raise the entrepreneurial competencies and intention of students? We used a pretest-post-test quasi-experimental design. Data were collected from 864 university students of Castilla & León (Spain), from 863 students (403 taking the programme and 460 in a control group). The results showed that students in the ‘programme’ group increased their competencies and intention towards self-employment, whereas students in the control group did not. The findings contribute to the theories of planned behaviour and to the literature of entrepreneurship education itself, by revealing the effect of specific benefits for the students derived from the entrepreneurship ‘programme’.     

Allocation of public funds to R&;D: a portfolio choice-styled decision model and a biotechnology case study

A decision problem—allocating public research and development (R&D) funding—is faced by a planner who has ambiguous knowledge of welfare effects of the various research areas. We model this as a reverse portfolio choice problem faced by a Bayesian decision-maker. Two elements of the planner’s inferential system are developed: a conditional distribution of welfare ‘returns’ on an allocation, given stated preferences of citizens for the different areas, and a minimum risk criterion for re-allocating these funds, given the performance of a status quo level of funding. A case study of Canadian public research funds expended on various applications of agricultural biotechnology is provided. The decision-making methodology can accommodate a variety of collective expenditure and resource allocation problems.     

Preventing mismatch answers in standardized survey interviews

Interaction analysis of question–answer sequences from a telephone survey shows that so-called mismatch answers, i.e. answers that do not correspond to the required answering format, are the most frequently occurring problematic verbal behavior. They also are likely to trigger suggestive interviewer probing. Explanations for the occurrence of mismatch answers concern cognitive and conversational factors. In both a non-experimental and an experimental study it was found that questions with formal response alternatives yield more mismatch answers than questions with colloquial response alternatives (i.e., words that are frequently used in ordinary conversations). Effects of the wording of questions were only found in the non-experimental study, indicating that formal question wording yields fewer mismatch answers than colloquial question wording. The findings suggest that, especially in case of questions that are formulated as agree–disagree opinion assertions, the chance of mismatch answers is highly reduced when colloquial response alternatives are used.     

Probing: A dangerous practice in social surveys?

It is widely accepted among survey researchers that the use of probes by their interviewers can lead to more intelligible, more complete, and more easily coded answers from respondents. Yet it is not hard to find examples of the use of probes in the survey research literature that have failed to produce demonstrably better data. Since the poor codability of respondents' answers is the product of a number of factors, merely training interviewers to be more skilful in their use of probes is unlikely to significantly improve the quality of survey data. If probes are to be used, they should be both standardized and used systematically. It would be better, however, to try to eliminate the need for interviewers to have to use probes. Two positive steps the researcher can take in this direction are: to define key terms properly, and to specify the response frameworks that respondents should adopt when formulating replies to particular questions.     

Economic designs of two-sided single and double screening procedures

Economic designs of single and double screening procedures for improving outgoing product quality based on two screening variables are presented for the case of two-sided specification limits. Two screening variables are observed simultaneously in the single screening procedure. In the double screening procedure, one variable is used first to make one of three decisions — accept, reject, or undecided — and after the first screening, the second variable is employed to screen the undecided items. It is assumed that the performance and the two screening variables are jointly normally distributed, and the deviation of the performance variable from the ‘ideal’ value causes dissatisfication to the consumers. Two quality cost functions — constant and quadratic — are considered. Cost models are constructed which involve screening inspection cost, and costs of accepted and rejected item. Methods of finding the optimal cutoff values are presented and a numerical example is given.     

Using a highly portable computer to gather qualitative social network data

We report on the use of a highly portable computer for collecting precise and accurate data concerning specific social network ties. Our experience confirmed that a highly portable computer, when loaded with a flexible database management program, is a useful tool for such a task. Data entry can be done during the interview and information retrieval can provide immediate and useful information for the respondents. The use of the computer with semistructured interviews changes the dynamics of the interview situation. A three-way interaction is created between the interviewers, the respondents and the information in the database. We found that there were great benefits from having two or three respondents in the session, and that it was desirable to have two interviewers.     

Order Effects in Batteries of Questions

Batteries of questions with identical response items are commonly used in survey research. This paper suggests that question order has the potential to cause systematic positive or negative bias on responses to all questions in a battery. Whilst question order effects have been studied for many decades, almost no attention has been given to this topic. The primary aim is to draw attention to this effect, to demonstrate its possible magnitude, and to discuss a range of mechanisms through which it might occur. These include satisficing, anchoring and cooperativeness. The effect seems apparent in the results of a recent survey. This was a survey of Emergency Department patients presenting to Wollongong Hospital (Australia) with apparently less urgent conditions in 2004. Two samples were taken. Question order was fixed in the first sample (n = 104; response rate RR2 = 94%), but randomised in the second sample (n = 46; response rate RR2 = 96%). Respondents were asked to indicate whether each of 18 reasons for presenting to the ED was a ‘very important reason’ a ‘moderately important reason’ or ‘not a reason’ The mean number of very important reasons selected was 56% higher in the first sample as compared to the second sample.