Qualitative Research in the Service of Understanding Learners and Users of Statistics

This paper seeks to add to a scholarly dialogue regarding the role and value of qualitative techniques in research on learning and using statistics. The paper briefly outlines some of the core assumptions of qualitative research methods, and presents four examples to illustrate selected qualitative methods that are used by educational researchers and service organizations. The discussion emphasizes the need to integrate quantitative and qualitative approaches in research on learners and users of statistics, and suggests that such integration may be needed to study emerging web-based communities of learners and users of statistics.     

The distinction between qualitative and quantitative research methods is problematic

The distinction between qualitative and quantitative research is abstract, very general and its value is usually taken for granted. In contrast, this article attempts to show that the distinction between qualitative and quantitative research is unclear, poor and therefore of limited value and that its popularity risks leading to unfortunate consequences. Various arguments are presented for this conclusion. For example, it is argued that the heterogeneity of different stand-points on important issues among qualitative researchers (for example with respect to the use of quantification and causal analysis) makes the distinction as such unstable. Moreover, the presence of substantial overlap between many features of qualitative and quantitative research often makes it difficult to separate qualitative and quantitative research. It is also shown that three obvious ways of making the distinction between qualitative and quantitative research are unsatisfactory. Use of the distinction may restrict creativity in the development of new research methods and create confusion and unnecessary work. In general, it may be preferable not to conceptualize research approaches at such abstract levels as done in the context of qualitative or quantitative approaches. Instead, it is suggested that it is more fruitful to discuss the pros and cons of specific research methods, preferably in the context of specific research problems.     

QUALITATIVE AND MIXED‐METHODS RESEARCH IN ECONOMICS: SURPRISING GROWTH,PROMISING FUTURE

Qualitative research in economics has traditionally been unimportant compared to quantitative work. Yet there has been a small explosion in use of quantitative approaches in the past 10–15 years, including ‘mixed‐methods’ projects which use qualitative and quantitative methods in combination. This paper surveys the growing use of qualitative methods in economics and closely related fields, aiming to provide economists with a useful roadmap through major sets of qualitative methods and how and why they are used. We review the growing body of economic research using qualitative approaches, emphasizing the gains from using qualitative‐ or mixed‐methods over traditional ‘closed‐ended’ approaches. It is argued that, although qualitative methods are often portrayed as less reliable, less accurate, less powerful and/or less credible than quantitative methods, in fact, the two sets of methods have their own strengths, and how much can be learned from one type of method or the other depends on specific issues that arise in studying the topic of interest. The central message of the paper is that well‐done qualitative work can provide scientifically valuable and intellectually helpful ways of adding to the stock of economic knowledge, especially when applied to research questions for which they are well suited.     

Towards a flexible online mixed method design with a feedback loop

The notion of mixed methods design relates to the research studies that combine qualitative and quantitative approaches. However, most of these studies are tailored to specific research problem in a particular study and are typically limited to a fixed sequence of qualitative and quantitative approaches (e.g. qualitative interviews followed by a survey or vice-versa). This limitation historically arises from time, cost and logistic restrictions. As an alternative, we develop an general extension of fixed mixed method design by introducing a flexible feedback-loop so that several phases can be combined in a flexible order. In practice, such designs are now increasingly feasible within an information-communication technology environment, where online respondents are readily available for immediate participation. An online experiment combining interactive series of web surveys and in-depth web interviews was performed to compare this approach with standard two-phase designs involving mixed methods. The costs, timing, quality of finiding and experiences of researchers were systematically evaluated. In summary, the proposed approach proved to be very beneficial.     

QUALITATIVE RESEARCH IN ORGANIZATIONAL BEHAVIOUR

Qualitative and quantitative approaches to the study of organizations are not mutually exclusive. Yet, over the last several years they have tended to become as such. Also, the interest in qualitative research of organizations seems to be renewing in the recent years. The present paper looks at some of the currently popular qualitative tools of organizational research emphasizing the complementary nature of qualitative and quantitative approaches in most settings.     

The use of mixed methods research in the field of behavioural sciences

Mixed methods research involves the combined use of quantitative and qualitative methods in the same research study, and it is becoming increasingly important in several scientific areas. The aim of this paper is to review and compare through a mixed methods multiple-case study the application of this methodology in three reputable behavioural science journals: the Journal of Organizational Behavior, Addictive Behaviors and Psicothema. A quantitative analysis was carried out to review all the papers published in these journals during the period 2003–2008 and classify them into two blocks: theoretical and empirical, with the latter being further subdivided into three subtypes (quantitative, qualitative and mixed). A qualitative analysis determined the main characteristics of the mixed methods studies identified, in order to describe in more detail the ways in which the two methods are combined based on their purpose, priority, implementation and research design. From the journals selected, a total of 1,958 articles were analysed, the majority of which corresponded to empirical studies, with only a small number referring to research that used mixed methods. Nonetheless, mixed methods research does appear in all the behavioural science journals studied within the period selected, showing a range of designs, where the sequential equal weight mixed methods research design seems to stand out.     

Qualitative methodology and its pitfalls in educational research

There is considerable controversy in educational research with respect to the use of qualitative and quantitative data and as to what constitutes scientific research. The objective of this study is to explore the degree to which in-service teachers understand the difference between qualitative/quantitative data and methods, validity/authenticity, generalization and how these can be used to solve problems faced by the teachers. The study is based on 84 participants who had registered for a 10-week course on Methodology of Investigation in Education, as part of their Master’s degree program. The course is based on 11 readings drawing on a history and philosophy of science perspective (positivism, constructivism, Popper, Kuhn, Lakatos). Course activities included written reports, class room discussions based on participants’ presentations, and written exams. Based on the results obtained it is concluded: (a) Most participants understood that the problem to be investigated precedes the method and determines the methodology to be used; (b) As all observations are theory-laden, it is preferable that interpretations based on both qualitative and quantitative data be allowed to compete in order to provide validity to our research findings; (c) The difference between validity and authenticity was controversial and most participants considered the need for interpreting data and hence favored authenticity; (d) Discussions led to the idea of ‘degrees of validity’ as both validity in the quantitative sense and authenticity in the qualitative sense, ultimately depend on critical appraisals of the community; (e) Generalizability of results obtained from qualitative studies was a controversial topic and most participants agreed that it is not desirable to generalize; (f) Discussions suggested an alternative: In both qualitative and quantitative research generalizability is possible, provided we are willing to grant that our conceptions/theories are not entirely grounded in empirical evidence but rather on the degree to which the community can uphold such a consensus; (g) Most teachers considered the use of participant observation in qualitative research as non-controversial. Class discussions led to the understanding that emphasizing observations may lead us to the Aristotelian ideal of empirical science; (h) Formulation of hypotheses, manipulation of variables, and the quest for causal variables was considered by many teachers to be equivalent to the scientific method. Discussions facilitated the understanding that this led to idealization and thus helped to reduce the complexity of a problem.     

Bayesian data augmentation methods for the synthesis of qualitative and quantitative research findings

The possible utility of Bayesian methods for the synthesis of qualitative and quantitative research has been repeatedly suggested but insufficiently investigated. In this project, we developed and used a Bayesian method for synthesis, with the goal of identifying factors that influence adherence to HIV medication regimens. We investigated the effect of 10 factors on adherence. Recognizing that not all factors were examined in all studies, we considered standard methods for dealing with missing data and chose a Bayesian data augmentation method. We were able to summarize, rank, and compare the effects of each of the 10 factors on medication adherence. This is a promising methodological development in the synthesis of qualitative and quantitative research.     

Mixed methods research synthesis: definition, framework, and potential

Literature on the combination of qualitative and quantitative research components at the primary empirical study level has recently accumulated exponentially. However, this combination is only rarely discussed and applied at the research synthesis level. The purpose of this paper is to explore the possible contribution of mixed methods research to the integration of qualitative and quantitative research at the synthesis level. In order to contribute to the methodology and utilization of mixed methods at the synthesis level, we present a framework to perform mixed methods research syntheses (MMRS). The presented classification framework can help to inform researchers intending to carry out MMRS, and to provide ideas for conceptualizing and developing those syntheses. We illustrate the use of this framework by applying it to the planning of MMRS on effectiveness studies concerning interventions for challenging behavior in persons with intellectual disabilities, presenting two hypothetical examples. Finally, we discuss possible strengths of MMRS and note some remaining challenges concerning the implementation of these syntheses.     

区域物流能力量化分析研究

文中对河北省衡水市区域物流能力的相关数据进行了调查,采用文献研究法、定性分析和定量分析法、理论研究与实际研究相结合法对调研所得数据进行分析。深入而系统的研究了区域物流系统的构建与模型探讨,在指标的定性与定量上对区域物流能力进行分析。并得出如下结论：2007~2011年的衡水市区域物流能力值除了在2010年有所下降外,在整体上还是保持了持续上升的态势。     

Generalization practices in qualitative research: a mixed methods case study

The purpose of this mixed methods case study was to examine the generalization practices in qualitative research published in a reputable qualitative journal. In order to accomplish this, all qualitative research articles published in Qualitative Report since its inception in 1990 (n =  273) were examined. A quantitative analysis of the all 125 empirical qualitative research articles revealed that a significant proportion (i.e., 29.6%) of studies involved generalizations beyond the underlying sample that were made inappropriately by the author(s). A qualitative analysis identified the types of over-generalizations that occurred, which included making general recommendations for future practice and providing general policy implications based only on a few cases. Thus, a significant proportion of articles published in Qualitative Report lack what we call interpretive consistency.     

A comparison of qualitatively and quantitatively driven analytic procedures of psychotherapeutic group sessions with deviant adolescents

The present paper deals with the therapy-related change in a group of deviant adolescents attending a therapeutic facility. The main issue of two studies reported here is to identify pattern of changes through different methods. The paper describes how the mixing of two different types of analysis may yield a more complex and complete view of the data. The analyses were performed at the same time: the first part (study 1), which was of a clinical–quantitative nature, involved picking out changerelated features through the analysis of the frequency of lemmas in the content of the sessions; the second part (study 2) had a qualitative orientation and traced the pattern of narrative themes in the sessions during the course of time. In both studies, the same data were analyzed at the same times, though each study adopted a different approach. While both approaches focused on change, the results brought out different aspects: the study that adopted the quantitative/clinical approach highlighted changerelated problems and proposed an explanation in terms of the “deviant peer effect”; by contrast, the qualitative study underlined the possibilities offered by change and viewed therapeutic change in terms of the shift in narrative themes from the detachment and lack of responsibility initially shown by the youths towards progressive maturation that may have been due to the efficacy of the therapy. The discussion of results brings us to consider the fuzzy set approach for future research into the field of criminology and group therapy. In conclusion, the paper shows that looking at the same data through two different methodological “lenses” yields different, and only apparently contradictory, conclusions.     

The Use of Triangulation in a Study of Refugee Well-Being

This article illustrates the use of triangulation in a study of refugee well-being. While the study was driven by a qualitative paradigm, information and data from qualitative and quantitative methods were integrated in order to fully explore, describe and understand the phenomena being studied. Hence, through triangulating information and data from an ethnographic interview, demographic and health questionnaires, and observations, the subjective and objective aspects of refugee experience and well-being were fully revealed. The advantages of triangulation, its pitfalls, and the controversy surrounding this methodology within the scientific health community are discussed. Finally, the author offers recommendations pertaining to the use of methodological triangulation for other investigators who might consider employing this design in future research.     

Integrating quality into quantity: survey research in the era of mixed methods

As widely recognized during the golden age of survey research thanks to the work of the Columbia school, the use of mixed strategies allows survey research to overcome its limitations by incorporating the advantages of qualitative approaches rather than seeking alternative methods. The need to re-think survey research before embarking on this course impelled the author to undertake a critical analysis of one of the survey’s most important assumptions, proposing a shift from standardization of stimulus to standardization of meanings in order to anchor the requirement of answer comparability on a more solid basis. This proposal for rapprochement with qualitative research is followed by a more detailed section in which the author distinguishes four different types of mixed survey strategies, combining two criteria (time order and function of qualitative procedures). The most significant parts of the constructed typology are then brought together in a model called the multilevel integrated survey approach. This methodological model is concretely illustrated in an empirical study of homophobic prejudice among teenagers. The example shows how in research practice analytical mixed strategies can be creatively combined in the same survey research design, contributing to improvements in data quality and the relevance of research findings.     

Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research

Health care research includes many studies that combine quantitative and qualitative methods. In this paper, we revisit the quantitative-qualitative debate and review the arguments for and against using mixed-methods. In addition, we discuss the implications stemming from our view, that the paradigms upon which the methods are based have a different view of reality and therefore a different view of the phenomenon under study. Because the two paradigms do not study the same phenomena, quantitative and qualitative methods cannot be combined for cross-validation or triangulation purposes. However, they can be combined for complementary purposes. Future standards for mixed-methods research should clearly reflect this recommendation.     

A common base for quality control criteria in quantitative and qualitative research

In the social sciences, several scientific paradigms are mutually isolated owing to their use of specific sets of methodological criteria and quality control procedures. In this article, the central hypothesis, to be tested by conceptual analysis and logical reasoning, is that recommended procedures for quality control in quantitative as well as qualitative research can be derived from a common base of regulative ideas. By ‘qualitative’, we mean the complex of ethnographic, anthropological, symbolic interactionist, ethnoscience and related approaches. A second goal is to demonstrate the use of regulative ideas as a parsimonious and fruitful base for a comparative analysis of methodological canons. Although our focus is on the comparison of quantitative and qualitative (or: naturalistic) research, we also pay attention to policy research as opposed to fundamental research.     

Building Better Theory by Bridging the Quantitative–Qualitative Divide*

abstract Qualitative methods for data collection and analysis are not mystical, but they are powerful, particularly when used to build new or refine existing theories. This article provides an introduction to qualitative methods and an overview of tactics for ensuring rigor in qualitative research useful for the novice researcher, as well as more experienced researchers interested in expanding their methodological repertoire or seeking guidance on how to evaluate qualitative research. We focus our discussion on the qualitative analytical technique of grounded theory building, and suggest that organizational research has much to gain by coupling of use of qualitative and quantitative research methods.     

Calculation of scale elasticities in DEA models: direct and indirect approaches

The qualitative characterisation of returns to scale in DEA has been a research issue the last decade. However, quantitative information provides the ultimate information. This paper presents two ways of obtaining numerical values of scale elasticity by an indirect approach using efficiency scores and dual variables for radial projections of inefficient points to the frontier, and a direct approach that is more general and powerful and directly evaluates numerically scale elasticity at any point on the DEA surface along intersections with planes. The direct and indirect approaches are compared using real data and a very high correspondence is found.     

Contextual content analysis

This article suggests one way to systematically code textual data for research. The approach utilizes computer content analysis to examine patterns of emphasized ideas in text as well as the social context or underlying perspective reflected in the text. A conceptual dictionary is used to organize word meanings. An extensive profile of word meanings is used to characterize and discriminate social contexts. Social contexts are analyzed in relation to four reference dimensions (traditional, practical, emotional and analytic) which are used in the social science literature. The approach is illustrated with five widely varying texts, analyzed with selected comparative data. This approach has been useful in many social science investigations to system-atically score open-ended textual information. Scores can be incorporated into quantitative analysis with other data, used as a guide to qualitative studies, and to help integrate strengths of quantitative and qualitative approaches to research.     

Empirical evaluation of educational interactive systems

Educational software systems are usually highly interactive systems that needs to be evaluated from two different perspectives, first as interactive software, mainly using usability and accessibility approaches, and second as an educational product that needs to be evaluated from its learning impact. The empirical evaluation methods may be quantitative, with a more deductive reasoning, or qualitative, with a more inductive reasoning, oriented, however mixed approaches have been used extensively because they combine the both main methods with a more pragmatic knowledge focusing on knowing the consequences or solving problems. The empirical evaluation of interactive educational software due to its own characteristics requires a mixed approach typically. This special issue is devoted to identify, share and valorise best practices and experiences, with a very solid modelling background, that are focused on the empirical evaluation of educational interactive systems with a special emphasis on mixed quantitative and qualitative approaches.