Towards a flexible online mixed method design with a feedback loop

The notion of mixed methods design relates to the research studies that combine qualitative and quantitative approaches. However, most of these studies are tailored to specific research problem in a particular study and are typically limited to a fixed sequence of qualitative and quantitative approaches (e.g. qualitative interviews followed by a survey or vice-versa). This limitation historically arises from time, cost and logistic restrictions. As an alternative, we develop an general extension of fixed mixed method design by introducing a flexible feedback-loop so that several phases can be combined in a flexible order. In practice, such designs are now increasingly feasible within an information-communication technology environment, where online respondents are readily available for immediate participation. An online experiment combining interactive series of web surveys and in-depth web interviews was performed to compare this approach with standard two-phase designs involving mixed methods. The costs, timing, quality of finiding and experiences of researchers were systematically evaluated. In summary, the proposed approach proved to be very beneficial.     

Using mixed methods for the analysis of individuals: a review of necessary and sufficient conditions and an application to welfare state attitudes

When studying individuals, when is the combination of qualitative and quantitative methods better than just one method alone? Whereas the debate in macro-level research, such as in political science about comparing nations, has made progress in identifying meaningful logics for a combination of methods, it is yet unclear how these logics can be applied to the study of individuals. Individual-level dynamics are in tendency less inert than those of nations or organisations. Therefore, a combination of methods is more difficult to justify in individual-level analysis since differences in measurement results could be due to changes in the dynamics rather than due to the application of different techniques. In contrast, the assumption of unit homogeneity seems to be more easily met for individuals than for countries or other higher-level aggregates, facilitating a comparison of like and like. First, this article presents a compilation of conditions scattered across the literature for the analysis of individuals, according to which a mixed-method is preferable to a single-method approach. Second, the application of these conditions is illustrated with an analysis of the impact of intergenerational relationships on welfare state attitudes in Germany on the basis of survey and focus group data.     

Alternative quality standards in qualitative research?

Qualitative researchers often use other principles for judging the quality of their study than quantitative researchers. This inhibits a straightforward assessment of the quality and comparability of different types of studies, as well as decision-making about their usefulness for further research and practice. In this article, we question the use of alternative criteria, and argue for one coherent and inclusive framework of quality criteria for both qualitative and quantitative studies. We developed such a framework, based on a comparison and operationalization of different criteria, and the elaboration of procedures to realize them. Its usability is demonstrated by the application in a qualitative case study research and a mixed-methods study. This framework enhances advancing and judging the basic quality of any type of scientific research, promoting the assessment of the quality and hence usability of studies for further research or decision-making about practice.     

QUALITATIVE AND MIXED‐METHODS RESEARCH IN ECONOMICS: SURPRISING GROWTH,PROMISING FUTURE

Qualitative research in economics has traditionally been unimportant compared to quantitative work. Yet there has been a small explosion in use of quantitative approaches in the past 10–15 years, including ‘mixed‐methods’ projects which use qualitative and quantitative methods in combination. This paper surveys the growing use of qualitative methods in economics and closely related fields, aiming to provide economists with a useful roadmap through major sets of qualitative methods and how and why they are used. We review the growing body of economic research using qualitative approaches, emphasizing the gains from using qualitative‐ or mixed‐methods over traditional ‘closed‐ended’ approaches. It is argued that, although qualitative methods are often portrayed as less reliable, less accurate, less powerful and/or less credible than quantitative methods, in fact, the two sets of methods have their own strengths, and how much can be learned from one type of method or the other depends on specific issues that arise in studying the topic of interest. The central message of the paper is that well‐done qualitative work can provide scientifically valuable and intellectually helpful ways of adding to the stock of economic knowledge, especially when applied to research questions for which they are well suited.     

Preserving Theoretical Divergence in Management Research: Why the Explanatory Potential of Qualitative Research Should Be Harnessed Rather than Suppressed

Despite the evident rise in the number of qualitative studies that are published in leading journals, I argue that the rich tradition and hallmark of qualitative research is under pressure. In recent years qualitative papers are increasingly being fashioned in the image of quantitative research, so much so that papers adopt ‘factor‐analytic’ styles of theorizing that have typically been the preserve of quantitative methods. This is a worrying trend as it leads to certain types of explanations dominating our field and at the expense of other viable forms of explanation. It also narrows the remit of qualitative research in general by channelling the theoretical contribution of qualitative studies in the direction of factor‐analytic propositional or variance models. In this article, I discuss the differences between the distinct types of theoretical explanations that are associated with quantitative and qualitative methods, survey the trend towards a quantitative ‘restyling’ of qualitative research, and elaborate its negative implications for our body of knowledge and for the state of management and organization theory.     

Guidance for deciding upon use of primary mixed methods studies in research synthesis: lessons learned in childhood trauma

When reviewing literature, mixed methods studies (MMS) are increasingly retrieved, yet it is unclear how they should be dealt with in a research synthesis. In this article we examine the inclusion of primary MMS in research synthesis, based on experiences with a meta-analysis (MA) and a qualitative evidence synthesis (QES) in childhood trauma. The aim is to offer guidance for reviewers in deciding upon the use of MMS. This review article examines (1) the qualitative component, (2) the quantitative component, as well as (3) the third component of combined yield for use in a MA, a QES or a mixed studies review. A systematic search for MMS in the field of childhood trauma from January 1980 to October 2011 resulted in twelve MMS. Eight qualitative components, six quantitative components and one combined yield could have been included in a MA or QES. Exclusion of qualitative components was due to insufficient quality. Quantitative components were excluded because sample sizes were too small, different outcome measures than ours were used or no adequate statistics were provided. Yield could not be included because it was absent, outside the scope or otherwise unspecified. Finally, we offer flow charts with clear steps to assist researchers in deciding upon the use of components of MMS. Our study demonstrates that MMS can cover new areas and therefore cannot be neglected in a research synthesis.     

Portfolio selection methodology for a nuclear project

It is difficult to evaluate complex, publicly sensitive nuclear project proposals. Scientists, subject matter experts, politicians, and citizens often differ on scope, budget, time, and quality priorities. There are numerous qualitative factors and quantitative variables. Experts could widely disagree on criteria or neglect to discriminate between seemingly identical alternatives. An ineffective portfolio selection method could lead to safety problems, budget overruns, or outright project failures. This case study develops and examines a mixed‐method, integrated qualitative and quantitative portfolio selection model, applied to a “tritium extraction facility” project concept.     

Managing aging workers: a mixed methods study on bundles of HR practices for aging workers

Since abilities and motives change with age and common human resource (HR) practices might be less suitable for aging employees, scholars and practitioners are currently challenged to find new ways of managing aging workers and motivating them to continue working. Therefore, this mixed methods study builds on literature on lifespan development and literature on HR practices in proposing four new bundles of HR practices for aging workers: accommodative, maintenance, utilization and development HR bundles. Since we draw on separate bodies of literature to propose new constructs, we use a mixed methods design in order to triangulate our findings. The proposed HR bundles were explored with a qualitative interview study among HR managers, line managers and employees in the Dutch construction sector, and tested with a quantitative survey study among Dutch government workers. Both studies confirmed that HR practices for aging workers can be classified into accommodative, maintenance, utilization and development HR bundles of practices.     

Mixed Methods Studies in Environmental Management Research: Prevalence,Purposes and Designs

Mixed methods research, that is, the combined use of quantitative and qualitative methods in the same study, is a popular approach in several fields. However, its use in the field of environmental management and sustainable development has not been examined. The authors review the application of mixed methods research in the journal Business Strategy and the Environment in the last ten years. A total of 340 articles have been read and reviewed, and 26 mixed methods studies have been identified. The main purposes and mixed methods designs are examined. Suggestions on why and how to use mixed methods research are offered, and recommendations are provided to guide future mixed methods studies to advance our understanding of environmental management topics. Copyright © 2014 John Wiley & Sons, Ltd and ERP Environment     

Million dollar personality: a systematic literature review on personality in crowdfunding

Expressed personality traits can play a pivotal role in convincing investors in crowdfunding. Our study answers the research question: What is the current body of knowledge regarding the relationship between personality factors and crowdfunding success and where are knowledge gaps where the literature is silent? In our literature review, we therefore analyze and categorize (1) the results provided by quantitative studies on the relationship between the personality of entrepreneurs and crowdfunding success and (2) the research gaps identified by the authors investigating personality in crowdfunding. We find that studies investigating the entrepreneur's personality, i.e. the Big Five, other baseline personality traits (self-efficacy, innovativeness, locus of control, and need for achievement) and the Dark Triad, find positive relationships between openness and crowdfunding success, while narcissism shows an inverted u-shaped relationship with crowdfunding success across articles. However, the effects of other personality traits on crowdfunding success are largely inconclusive. Further, we identify four main gaps in the literature. First, future studies should examine non-linear relationships between expressed personality traits and crowdfunding success. Second, there is a need for more studies that employ different methods like qualitative or mixed-method approaches. Third, replication studies in similar and different contexts are urgently needed. Fourth, a plurality of personality perspectives would strengthen future research (e.g., investor perspective, third party perspective). To our knowledge this is the first literature review of personality traits in crowdfunding. Our work aims to enrich our understanding of individual-level components in the underexplored alternative finance market.

     

Auditing Quality of Research in Social Sciences

A growing body of studies involves complex research processes facing many interpretations and iterations during the analyses. Complex research generally has an explorative in-depth qualitative nature. Because these studies rely less on standardized procedures of data gathering and analysis, it is often not clear how quality was insured or assured. However, one can not easily find techniques that are suitable for such complex research processes to assess the quality of the study. In this paper, we discuss and present a suitable validation procedure. We first discuss how ‘diagnosing’ quality involves three generic criteria. Next, we present findings of previous research in possible procedures to assure the quality of research in social sciences. We introduce the audit procedure designed by Halpern [(1983) Auditing Naturalistic Inquiries: The Development and Application of a Model. Unpublished doctoral dissertation, Indiana University] we found an appropriate starting point for a suitable procedure for quality judgment. Subsequently, we will present a redesign of the original procedure, with according guidelines for the researcher (the auditee) and for the evaluator of the quality of the study (the auditor). With that design, we aim to enable researchers to bring forward their explorative qualitative studies as stronger and more equally valuable to studies that can rely on standardized procedures.     

Contextual content analysis

This article suggests one way to systematically code textual data for research. The approach utilizes computer content analysis to examine patterns of emphasized ideas in text as well as the social context or underlying perspective reflected in the text. A conceptual dictionary is used to organize word meanings. An extensive profile of word meanings is used to characterize and discriminate social contexts. Social contexts are analyzed in relation to four reference dimensions (traditional, practical, emotional and analytic) which are used in the social science literature. The approach is illustrated with five widely varying texts, analyzed with selected comparative data. This approach has been useful in many social science investigations to system-atically score open-ended textual information. Scores can be incorporated into quantitative analysis with other data, used as a guide to qualitative studies, and to help integrate strengths of quantitative and qualitative approaches to research.     

Taking the “Q” Out of Research: Teaching Research Methodology Courses Without the Divide Between Quantitative and Qualitative Paradigms

The purpose of this paper is to provide evidence that the debate between quantitative and qualitative is divisive and, hence, counterproductive for advancing the social and behavioral science field. We advocate that all graduate students learn to utilize and to appreciate both quantitative and qualitative research methodologies. As such, students will develop into pragmatist researchers who are able to utilize both quantitative and qualitative techniques when conducting research. We contend that the best way to accomplish this is by eliminating quantitative research methodology and qualitative research methodology courses from curricula and replacing these with research methodology courses at different levels that simultaneously teach both quantitative and qualitative techniques within a mixed methodological framework.An earlier version of this article received the 2003 Southwest Educational Research Association (SERA) Outstanding Paper Award.     

Quality in health care: possibilities and limitations of quantitative research instruments among health care users

Quality in health care has traditionally been dominated by medical profession, where patients’ opinions were labelled as lay evaluation. Patients’ views and opinions are important because they give us an insight into dimensions of quality that are not evaluated by medical profession and often seem to be more important. In health care quantitative methodology is often used to address these quality dimensions and introduce patients’ views and opinions. There are various benefits using quantitative research instruments, such as a detailed analysis of the importance of various quality dimensions for patients and an analysis of factors influencing patients’ satisfaction. On the other hand serious deficiencies can be tackled too, that are usually dealt with qualitative research instruments, because they go deeper into people’s motives and feelings. However, health care service is specific—it is very important to patients (health is one of the most important values), but their participation in health care service is rather low. They also don’t always say and do what they mean. In such a context combination of quantitative and qualitative research instrument does not give satisfactory answers. The importance of complaints is stressed and rewards for taking them seriously and acting upon them is discussed.     

A common base for quality control criteria in quantitative and qualitative research

In the social sciences, several scientific paradigms are mutually isolated owing to their use of specific sets of methodological criteria and quality control procedures. In this article, the central hypothesis, to be tested by conceptual analysis and logical reasoning, is that recommended procedures for quality control in quantitative as well as qualitative research can be derived from a common base of regulative ideas. By ‘qualitative’, we mean the complex of ethnographic, anthropological, symbolic interactionist, ethnoscience and related approaches. A second goal is to demonstrate the use of regulative ideas as a parsimonious and fruitful base for a comparative analysis of methodological canons. Although our focus is on the comparison of quantitative and qualitative (or: naturalistic) research, we also pay attention to policy research as opposed to fundamental research.     

Methodological criteria for the internal validity and utility of practice oriented research

For conducting practice oriented research different research strategies are available. In this paper a typology is developed for differentiating between practice oriented research strategies that are data based and participatory research strategies. The former category involves quantitative and qualitative data based research strategies. The latter category, participatory research strategies, includes knowledge based research and practice based research. At present, methodological criteria for assessing the quality of practice oriented research heavily rely on those developed for theory oriented research, in particular internal and external validity and reliability. However, we argue that for assessing the results of practice oriented research other criteria are necessary. In this paper, methodological criteria are formulated for evaluating the internal validity and practical utility of practice oriented research with the help of a Delphi study using research methodologists as experts. They agree upon the criteria of verifiability, comprehensibility and acceptance of the results, as well as holism. Moreover, different categories of participatory and data based research strategies are compared to these criteria. Practice based research and qualitative data based research are best equipped to fulfill these criteria. These findings may enable researchers to make a more deliberate choice for a specific research strategy in practice oriented research.     

Can Findings of Qualitative Research in Education be Generalized?

Most qualitative researchers do not recommend generalization from qualitative studies, as this research is not based on random samples and statistical controls. The objective of this study is to explore the degree to which in-service teachers understand the controversial aspects of generalization in both qualitative and quantitative educational research and as to how this can facilitate problems faced by the teachers in the classroom. The study is based on 83 participants who had registered for a 10-week course on ‘Methodology of Investigation in Education’ as part of their Master’s degree program. The course is based on 11 readings drawing on a philosophy of science perspective (positivism, constructivism, Popper, Kuhn, Lakatos). Course activities included written reports, class room discussions based on participants’ presentations, and written exams. Based on the results obtained it is concluded: (1) almost 91% of the teachers agreed that external generalization in a different social context is feasible; (2) almost 63% of the participants used a fairly inconsistent approach, that is in a theoretical context agreed that qualitative research cannot be generalized and still when asked with respect to the experience of two particular teachers, agreed that generalization was possible; (3) almost 28% of the participants used a consistent approach. Some of the reasons provided by the participants as to why generalization was feasible are discussed. An analogy is drawn with respect to Piaget’s methodology, viz., it was not based on random samples or statistical treatments and still his oeuvre has been generalized (criticisms not withstanding) in both the psychology and educational literature.     

Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research

Health care research includes many studies that combine quantitative and qualitative methods. In this paper, we revisit the quantitative-qualitative debate and review the arguments for and against using mixed-methods. In addition, we discuss the implications stemming from our view, that the paradigms upon which the methods are based have a different view of reality and therefore a different view of the phenomenon under study. Because the two paradigms do not study the same phenomena, quantitative and qualitative methods cannot be combined for cross-validation or triangulation purposes. However, they can be combined for complementary purposes. Future standards for mixed-methods research should clearly reflect this recommendation.