Preventing mismatch answers in standardized survey interviews

Interaction analysis of question–answer sequences from a telephone survey shows that so-called mismatch answers, i.e. answers that do not correspond to the required answering format, are the most frequently occurring problematic verbal behavior. They also are likely to trigger suggestive interviewer probing. Explanations for the occurrence of mismatch answers concern cognitive and conversational factors. In both a non-experimental and an experimental study it was found that questions with formal response alternatives yield more mismatch answers than questions with colloquial response alternatives (i.e., words that are frequently used in ordinary conversations). Effects of the wording of questions were only found in the non-experimental study, indicating that formal question wording yields fewer mismatch answers than colloquial question wording. The findings suggest that, especially in case of questions that are formulated as agree–disagree opinion assertions, the chance of mismatch answers is highly reduced when colloquial response alternatives are used.     

Question-Answer Sequences in Survey-Interviews

Interaction analysis was used to analyze a total of 14,265 question–answer sequences of (Q-A Sequences) 80 questions that originated from two face-to-face and three telephone surveys. The analysis was directed towards the causes and effects of particular interactional problems. Our results showed that problematic respondent behavior is affected by the questionnaire design, whereas inadequate interviewer behavior is affected by respondent behavior, rather than directly by the questionnaire design. Two surveys used questions for which validating information was available. It appeared that the occurrence of such irregularities of interviewer and respondent behavior was related to the validity of the eventual responses. Explanations for the occurrence of problematic respondent behavior were proposed, concerning both cognitive and conversational factors, related to the wording of questions and response alternatives. Part of this paper was presented at the International Conference on Questionnaire Development, Evaluation and Testing Methods, 14–17 November 2002, Charleston, South Carolina.     

Evaluation of the convergence between “self-reporters” and “proxies” in a disability questionnaire by means of behaviour coding method

Household surveys often require including proxy reporters to obtain information about other household members who cannot be interviewed. The participation of proxies can undermine survey data quality due to the fact that proxies must respond to questions thinking about other people. The objectives of the present study were to analyze the behaviour of proxy reporters and evaluate the convergence between the answers given by proxies and self-reporters by means of behaviour coding. This improves the evaluation of convergence, since only adequate (i.e., interpretable) answers given by both types of informant are taken into account. Responses to a disability questionnaire employed by an official statistical institute were analyzed. The questionnaire includes 11 questions about different limitations related to everyday activities. 16 self-reporter and 16 proxies formed 16 couples whose members lived together and supported a direct family relation. The results show a high percentage (52%) of convergence between both types of informant, although fluctuating across the questions and the couples. Proxies showed relatively more adequate behaviour during the interaction than self-reporters. From this we conclude that proxies can be considered at least as good informants as self-reporters from an interviewer-respondent interaction perspective. Future research should address the impact of proxy responses on survey validity.