Meta‐Analytic Research in Management: Contemporary Approaches,Unresolved Controversies,and Rising Standards

Early meta‐analyses in management research sought primarily to resolve seemingly conflicting findings by estimating a relationship’s population‐level effect size. Since then, management researchers have adopted increasingly sophisticated approaches that permit new theorizing, testing and comparing sophisticated models, and identifying boundary conditions. We summarize three of these approaches – i.e., qualitative meta‐analysis (QMA), meta‐analytic structural equation modeling (MASEM), and meta‐analytic regression analysis (MARA) – along with the special issue papers that adopt each approach. We conclude by raising three unresolved controversies that we believe deserve more attention and by offering our thoughts about how to maximize a meta‐analytic study’s chances for publication and impact.     

Organizational Controls and Performance Outcomes: A Meta‐Analytic Assessment and Extension

Managing employees and external partners effectively has been a primary concern for organizations and their managers. Many studies have investigated the effectiveness of organizational controls in a wide variety of contexts. Using organizational controls literature that discriminates among outcome, behaviour, and clan control, this study synthesizes the research on the effectiveness of these controls. In particular, the study examines 23,839 organizational controls–performance relationships from 120 independent samples, and tests several new hypotheses using advanced meta‐analytic methods. The results indicate that outcome, behaviour, and clan controls generally enhance performance, with each control having a distinct performance effect. Our analysis also demonstrates that controls function as complements to one another. This finding indicates that one form of control increases the effectiveness of other forms of control. We also examine the organizational controls–performance relationships across various contexts, and our results show that they vary according to the type of task. The paper concludes with a discussion on the theoretical and managerial implications of these findings.     

THE EURO'S TRADE EFFECT: A META‐ANALYSIS

Many studies have estimated the trade effect of the euro, but their results vary greatly. This meta‐analysis collects 3323 estimates of the euro effect along with 28 characteristics of estimation design from almost 60 studies and quantitatively examines the literature. The results show evidence of publication bias, but they also suggest that the bias decreases over time. After correcting for the bias, the meta‐analysis shows that the literature is consistent with an effect ranging between 2% and 6%. The results from Bayesian model averaging, which takes into account model uncertainty, show that the differences among estimates are systematically driven by data sources, data structure, control variables, and estimation techniques. The mean reported estimate of the euro's trade effect conditional on best‐practice approach is 3%, but is not statistically different from zero.     

A Process Perspective on Organizational Failure: A Qualitative Meta‐Analysis

An important stream of the organizational failure literature has proposed process models to describe how firms fail. Despite much progress, this stream is currently at a crossroads. Previous process models try to capture how failure unfolds in singular models that describe organizational failure as the result of either inertia or extremism or as a mixture of both. However, it remains unclear how these competing explanations are related and what underlying mechanisms explain why organizational failure processes unfold as they do. We address these issues by examining failure processes using a qualitative meta‐analysis research design. The qualitative meta‐analysis allows us to analyse and synthesize the wealth of previously published single‐case studies in order to develop process models of organizational failure. The most salient finding of our analysis is that failure processes converge around four distinct process archetypes, which we name imperialist, laggard, villain, and politicized. Each process archetype can be explained by the interplay of distinct rigidity and conflict mechanisms. Differentiating the four process archetypes and explaining the underlying mechanisms helps to resolve some contradictions in the previous failure process literature.     

Understanding variation in the relationship between environmental management practices and firm performance across studies: A meta‐analytic review

The relationship between environmental management practices and firm performance has continuously received much attention in academic research. As existing literature on the link of the two constructs is characterized by heterogeneous research characteristics and mixed empirical evidence, one interesting question arises: Do research characteristics affect the magnitude and direction of the relationship between environmental management practices and firm performance? Amassing a database of 92 studies, 199 effect sizes, and 72,258 firms, we examine this question by conducting a meta‐analysis. The characteristics investigated in this study include item number of performance measures, year of data collection, industry type, economic development, and Hofstede's five cultural dimensions. A meta‐regression reveals that environmental management practices are positively associated with firm performance and that the degree of this association depends on these contexts. Specifically, the environmental management practices–performance relationship increases with year of data collection, is stronger with multiple item measures, and is higher for firms in developed countries and in cultures characterized by high power distance, low individualism, low uncertainty avoidance, and low long‐term orientation.     

Does the Diversification–Firm Performance Relationship Change Over Time? A Meta‐Analytical Review

We study the relationship between diversification and firm performance in the context of the decline in levels of diversification over time. We argue that the pressure to reduce diversification may have more strongly affected those firms whose diversification strategies were most detrimental to firm performance. We employ meta‐analytical regression (MARA) in order to test our hypotheses, using a total of 267 primary studies containing 387 effect sizes based on 150,000 firm‐level observations from over 60 years of research on the diversification–firm performance relationship. The findings suggest that levels of unrelated diversification have decreased, whereas levels of related diversification have increased since the mid‐1990s, following an initial decrease in the 1970s and 1980s. Furthermore, we find that the relationship between unrelated diversification and firm performance has improved significantly over time, whereas the relationship between related diversification and performance has remained relatively stable.     

Innovation Offshoring,Institutional Context and Innovation Performance: A Meta‐Analysis

Innovation offshoring (IO) has become a widespread management practice. Yet, evidence on the performance implications is inconsistent, and scattered across disciplines and contexts. We argue that the benefits firms can derive from IO depend on the institutional environment at home. Drawing on recent work on institutional theory in international business, we explore institutions that facilitate reverse knowledge transfer and/or institutional arbitrage with respect to innovation‐related activities. The results of our meta‐analysis that synthesizes evidence from 48 samples show that IO is related positively to innovation performance. As predicted, this relationship is moderated by differences in the institutional environments across countries. Specifically, when national innovation systems are weak at home, IO appears to enable institutional arbitrage strategy whereas Confucian cultures enable more effective reverse knowledge transfer. However, contrary to our expectations, the beneficial effects of IO appear to have diminished over time.     

Employee ownership and firm performance: a meta‐analysis

Employee ownership has been an area of significant practitioner and academic interest for the past four decades. Yet, empirical results on the relationship between employee ownership and firm performance remain mixed. To aggregate findings and provide potential direction for future theoretical development, we conducted a meta‐analysis of 102 samples representing 56,984 firms. Employee ownership has a small, but positive and statistically significant relation to firm performance ( = 0.04). The effect is generally positive for studies with different sampling designs (samples assessing change in performance pre‐employee–post‐employee ownership adoption or samples on firms with employee ownership), different performance operationalisation (efficiency or growth) and firm type (publicly held or privately held). Suggesting benefits of employee ownership in a variety of contexts, we found no differences in effects on performance in publicly held versus privately held firms, stock or stock option‐based ownership plans or differences in effects across different firm sizes (i.e. number of employees). We do find that the effect of employee ownership on performance has increased in studies over time and that studies with samples from outside the USA report stronger effects than those within. We also find little to no evidence of publication bias.     

A meta‐analytic investigation of the relationship between HRM bundles and firm performance

Human resource management bundles consisting of multiple complementary practices are typically considered superior to individual best practices in influencing firm performance. This study investigates the relationship between three such bundles (empowerment, motivation, and skill‐enhancing) and business outcomes (retention, operating performance, financial performance, and overall performance ratings). A meta‐analysis of 239 effect sizes derived from 65 studies reveals that HRM bundles have significantly larger magnitudes of effects than their constituent individual practices, are positively related to business outcomes, and display effect sizes that are comparable to or larger than those of high‐performance work systems. These findings reaffirm the case for firm‐level investments in synergistic HRM combinations and highlight the importance of investing in complementary practices. © 2009 Wiley Periodicals, Inc.     

Training and organisational performance: A meta‐analysis of temporal,institutional and organisational context moderators

Drawing on systems theory, we conducted a moderated meta‐analysis of the training and organisational performance relationship using 119 primary studies. We examined the moderating effects of quality versus quantity of training, time, institutional and organisational context factors in the relationship between training and organisational performance. Our findings reveal that training is positively and directly related to organisational performance with no statistically significant difference between measures of training quality and quantity. We found that the relationship was stronger over time and that country performance orientation and country labour cost moderate the training and organisational performance relationship. We found no evidence for the moderating effects of the three organisational context moderators we examined (i.e. industry sector, organisational size and technology intensity). Finally, our results reveal that training type (i.e. general or firm‐specific) does not moderate the training and organisational performance relationship.     

HOW DO RATING AGENCIES’ DECISIONS IMPACT STOCK MARKETS? A META‐ANALYSIS

The purpose of this study is to examine how credit rating agencies’ decisions impact the stock market using a systematic and quantitative review of existing empirical studies. Specifically, we employ a meta‐regression analysis (MRA) to investigate the extent and nature of the effect of rating agencies’ decisions on the stock market. We survey 62 studies published between 1978 and 2015. Our first finding is that the cumulative average abnormal returns calculated from this empirical literature are affected by publication bias. After controlling for publication bias, the main findings of our meta‐analysis indicate that negative rating decisions cause statistically significant negative abnormal returns. This evidence suggests an informational effect. Our results also indicate that positive rating decisions do not have a significant effect. Finally, the MRA results reveal the importance of several factors related to primary study design, as well as to the nature of the data.     

META‐ANALYSIS OF ECONOMICS RESEARCH REPORTING GUIDELINES

Meta‐regression analysis (MRA) can provide objective and comprehensive summaries of economics research. Their use has grown rapidly over the last few decades. To improve transparency and to raise the quality of MRA, the meta‐analysis of economics research‐network (MAER‐Net) has created the below reporting guidelines. Future meta‐analyses in economics will be expected to follow these guidelines or give valid reasons why a meta‐analysis must deviate from them.     

Which Resources Matter How and Where? A Meta‐Analysis on Firms’ Foreign Establishment Mode Choice

Firm resources play an important role in explaining the foreign establishment mode choice. However, a theoretical framework that simultaneously a) differentiates between heterogeneous types of resources, b) provides a rationale that consistently explains how firms adapt their establishment mode choice to their resource base, and c) reflects on contextual factors that intervene in the consistency of the direct relationships is largely missing. The present paper draws on the resource‐based view (RBV) to develop such a framework. More specifically, we study the effect of knowledge‐based and experience‐based resources on a firm's establishment mode choice and the moderating impact of cultural distance on these relationships. We validate our theoretical framework by a meta‐analysis (integrating 31 studies with 13,559 establishment mode choices) and find that knowledge‐based resources enhance a firm's propensity for greenfield investments, while experience‐based resources more likely lead to acquisitions. Further, cultural distance increases knowledge‐intensive firms’ preference for greenfield investments.     

Illuminating the ‘Face’ of Justice: A Meta‐Analytic Examination of Leadership and Organizational Justice

A significant body of research has described effective leader behaviours and has connected these behaviours to positive employee outcomes. However, this research has yet to be systematically integrated with organizational justice research to describe how leader behaviours inform justice perceptions. Therefore, we conduct a meta‐analysis (k = 166, N = 46,034) to investigate how three types of leader behaviours (task, relational, and change) inform four dimensions of organizational justice (procedural, distributive, interpersonal, and informational) referenced to the leader and to the organization. Further, we examine the joint impact of leader behaviours and justice perceptions on social exchange quality (i.e., leader–member exchange), task performance, and job satisfaction. Our results suggest that leader behaviours differentially inform leader‐ and organization‐focused justice perceptions, and the joint effect of leader behaviours and justice perceptions offer more nuanced explanations for outcomes.     

Dynamic Capabilities and Organizational Performance: A Meta‐Analytic Evaluation and Extension

We move the dynamic capabilities view (DCV) forward in two important ways by meta‐analysing prior empirical studies. First, we evaluate the two core theoretical tenets of the DCV: (1) Dynamic capabilities are positively related to performance, and (2) this relationship is stronger in industries with higher levels of technological dynamism. We find support for the former (r_c = 0.296) but not for the latter, though results suggest the existence of moderators. Second, we theorize and demonstrate empirically that higher‐order dynamic capabilities are more strongly related to performance than lower‐order dynamic capabilities, lower‐order dynamic capabilities partially mediate the relationship between higher‐order dynamic capabilities and performance, and dynamic capabilities contribute more to performance in developing economies than in developed economies. These findings illustrate how the nature of the dynamic capability and the economic context in which it is utilized shape its value, thus offering a more nuanced conceptualization of the dynamic capabilities‐performance relationship.     

WHICH AGGLOMERATION EXTERNALITIES MATTER MOST AND WHY?

This paper revisits the ongoing discussion on the importance of agglomeration externalities – specifically specialization, diversity and competition effects – that may contribute to innovation, productivity and urban employment growth. Previous meta‐analyses suggested that the evidence on agglomeration externalities is strongly context‐specific. Expanding an earlier analysis of 31 articles, we seek to draw in this paper more robust conclusions by means of the statistical evidence for agglomeration externalities presented in 73 scientific articles, all building on the seminal work of Glaeser et al. (1992). Our results confirm that the heterogeneity among studies is huge and can only be partially accounted for by means of an ordered probit analysis. Additionally, some evidence of publication bias is found. We conclude that the conventional lines of inquiry in this literature may now have reached strongly diminishing returns. New lines of inquiry, using rich micro‐level data on firms and workers, dynamic general equilibrium models at the macro level, more attention for spatial and temporal variation in the impacts of agglomeration, and further investigations into the spatial scope of externalities are promising avenues for further research that can enhance our understanding of how agglomeration externalities continue to fuel our increasingly urbanized world.     

FDI AND TAXATION: A META‐STUDY

Abstract In this paper we extend former meta‐analyses on FDI and taxation in three ways. First, we add 16 recent publications. Second, we code additional meta‐regressor variables addressing important issues in research on FDI and taxation. Third, we refer to the sophisticated meta‐analytical methodology and present a coherent strategy to choose the meta‐regression estimator most suitable for the meta‐data at hand. As compared to prior surveys, the meta‐analysis is thus based on a much broader methodological basis and a considerably richer meta‐data set. The median tax semi‐elasticity of FDI based on 704 primary estimates is 2.49 in absolute terms. The precision weighted average of the full sample of semi‐elasticities is 2.55, again in absolute terms. Moreover, our meta‐analysis shows that there is a publication bias in the primary literature. Meta‐regressions show that studies based on aggregate data report systematically larger semi‐elasticities than firm‐level analyses, that integrating bilateral tax regulations into effective tax rates leads to more effective measurement of adverse tax incentives on foreign investment, and that tax effects are not compensated by public spending.     

ALCOHOL CONSUMPTION AND HUMAN CAPITAL: A RETROSPECTIVE STUDY OF THE LITERATURE

Abstract Grossman proposed an individual's health can be viewed as one aspect of their human capital. Following this line of thought a number of recent papers have reported a positive impact of alcohol consumption on earnings. The rationale for the existence of such a relationship is the positive impact of alcohol on physical and mental health. We conduct a meta‐analysis to determine whether such factors as: the estimation technique, the presence of ex‐drinkers in the sample, possible sample selection bias and publication bias may all contribute to these findings. An additional suggestion for the positive relationship between alcohol and wages is the presence of a common set of personality traits that determines drinking behaviour and also leads to higher earnings. We examine this relationship by reviewing the literature that investigates if the personality influencing aspects of alcohol consumption influences measures of human capital. We also survey the significant body of research that has examined how alcohol consumption has been found to influence educational outcomes and the work force participation of problem drinkers.     

Organizational Citizenship Behavior and Role Breadth: A Meta‐Analytic and Cross‐Cultural Analysis

We provide a meta‐analysis of the empirical literature concerning role breadth, defined as the degree to which employees consider organizational citizenship behavior (OCB) to be an inherent part of their job. Results based on a combined sample size of 9,222 showed: (a) Confucian Asians consider OCB as part of their job to a greater extent than do their Anglo counterparts; (b) affiliative kinds of OCB (e.g., helping, conscientiousness, and courtesy) are more likely to be considered part of one's job than are change‐oriented OCB (e.g., voice, taking charge, and initiative); and (c) OCB‐inclusive role breadth correlates strongly with OCB (r_c = .43). The implications of these findings for human resources practice, such as competency modeling, employee selection and training, organizational rewards, and employee‐employer/supervisor relations, are discussed. © 2013 Wiley Periodicals, Inc.     

WHAT'S TO KNOW? PUZZLES IN THE LITERATURE ON THE VALUE OF STATISTICAL LIFE

Abstract Notwithstanding the general acceptance of the value of statistical life (VSL) estimates for policy assessment purposes, several important unresolved issues remain. First, the results from revealed preference studies are systematically higher than those from stated preference studies, potentially limiting the usefulness of stated preference studies in generalizing the VSL estimates to different populations and kinds of risks. Second, extrapolating the results of meta‐analyses to project the VSL for different population groups requires that such generalization be reflective of the underlying economic content of what average VSL estimates reflect. Third, government agencies within and across countries place differing emphasis on types of VSL studies as well as differing reliance on individual studies versus meta‐analyses. Usually, there is no justification provided for the chosen approach.