SUCCESSFUL IN THE LONG RUN: A META‐REGRESSION ANALYSIS OF PERSISTENT FIRM PROFITS

We study the profit persistence literature by applying meta‐regression analysis (MRA) to a set of 36 empirical papers, which analyze the persistence of abnormal firm profits over time. The analyzed literature provides evidence for a mediocre degree of persistence in abnormal profits. An initial analysis of the distribution of reported profit persistence estimates reveals some degree of excess variation. This points toward publication bias that favors significant results independent of their algebraic sign. The MRA, however, reveals that publication bias is particularly favoring results that indicate profit persistence and thus contradict the neoclassical model of perfect competition. Moreover, the MRA enables to control for heterogeneity driven by the study design. We find that the analyzed country (developing vs. developed), the applied econometric approach, as well as the analyzed period of time are significant drivers of heterogeneity in reported persistence estimates.     

HOW DO RATING AGENCIES’ DECISIONS IMPACT STOCK MARKETS? A META‐ANALYSIS

The purpose of this study is to examine how credit rating agencies’ decisions impact the stock market using a systematic and quantitative review of existing empirical studies. Specifically, we employ a meta‐regression analysis (MRA) to investigate the extent and nature of the effect of rating agencies’ decisions on the stock market. We survey 62 studies published between 1978 and 2015. Our first finding is that the cumulative average abnormal returns calculated from this empirical literature are affected by publication bias. After controlling for publication bias, the main findings of our meta‐analysis indicate that negative rating decisions cause statistically significant negative abnormal returns. This evidence suggests an informational effect. Our results also indicate that positive rating decisions do not have a significant effect. Finally, the MRA results reveal the importance of several factors related to primary study design, as well as to the nature of the data.     

ARE ALL ECONOMIC FACTS GREATLY EXAGGERATED? THEORY COMPETITION AND SELECTIVITY

Abstract ..There is growing concern and mounting evidence of selectivity in empirical economics. Most empirical economic literatures have a truncated distribution of results. The aim of this paper is to explore the link between publication selectivity and theory contests. This link is confirmed through the analysis of 87 distinct empirical economics literatures, involving more than three and a half thousand separate empirical studies, using objective measures of both selectivity and contests. Our meta–meta‐analysis shows that publication selection is widespread, but not universal. It distorts scientific inference with potentially adverse effects on policy making, but competition and debate between rival theories reduces this selectivity and thereby improves economic inference.     

WHAT'S TO KNOW ABOUT TIME USE?

Stimulated by the availability of large sets of microeconomic data, research on the economics of time use has been a growth industry in the past 20 years. That growth has included studies that have focused on the effect of people's value of time; on the mix of nonmarket activities that they undertake; on the interactions of spouses’ choices of time use; on the valuation of nonmarket time, and on the timing of nonmarket and market activities. By laying out these research questions and indicating their importance, this essay provides a framework for a series of meta‐analyses.     

WHAT'S TO KNOW ABOUT LABORATORY EXPERIMENTATION IN ECONOMICS?

Abstract Experimental economics has grown as a discipline from near non‐existence 50 years ago to a full‐fledged field within economics in the present. Much of experimental economics research involves experimental methods as a tool, applied to problems in other fields of economics. However, some of this research is inward looking, focusing on questions of the methodology of experimental economics. In this note, I briefly discuss two methodological issues in experimental economics that might benefit from meta‐analysis: the pool from which experimental participants are drawn (university undergraduate students versus other populations) and the scale of monetary incentives faced by participants (large, small or hypothetical).     

FDI AND TAXATION: A META‐STUDY

Abstract In this paper we extend former meta‐analyses on FDI and taxation in three ways. First, we add 16 recent publications. Second, we code additional meta‐regressor variables addressing important issues in research on FDI and taxation. Third, we refer to the sophisticated meta‐analytical methodology and present a coherent strategy to choose the meta‐regression estimator most suitable for the meta‐data at hand. As compared to prior surveys, the meta‐analysis is thus based on a much broader methodological basis and a considerably richer meta‐data set. The median tax semi‐elasticity of FDI based on 704 primary estimates is 2.49 in absolute terms. The precision weighted average of the full sample of semi‐elasticities is 2.55, again in absolute terms. Moreover, our meta‐analysis shows that there is a publication bias in the primary literature. Meta‐regressions show that studies based on aggregate data report systematically larger semi‐elasticities than firm‐level analyses, that integrating bilateral tax regulations into effective tax rates leads to more effective measurement of adverse tax incentives on foreign investment, and that tax effects are not compensated by public spending.     

ALCOHOL MARKETING,ADOLESCENT DRINKING AND PUBLICATION BIAS IN LONGITUDINAL STUDIES: A CRITICAL SURVEY USING META‐ANALYSIS

Abstract This paper presents a meta‐analysis of prospective cohort (longitudinal) studies of alcohol marketing and adolescent drinking, which accounts for publication bias. The paper provides a summary of 12 primary studies of the marketing–drinking relationship. Each primary study surveyed a sample of youth to determine baseline drinking status and marketing exposure, and re‐surveyed the youth to determine subsequent drinking outcomes. Logistic analyses provide estimates of the odds ratio for effects of baseline marketing variables on adolescent drinking at follow‐up. Using meta‐regression analysis, two samples are examined in this paper: 23 effect‐size estimates for drinking onset (initiation); and 40 estimates for other drinking behaviours (frequency, amount, bingeing). Marketing variables include ads in mass media, promotion portrayals, brand recognition and subjective evaluations by survey respondents. Publication bias is assessed using funnel plots that account for ‘missing’ studies, bivariate regressions and multivariate meta‐regressions that account for primary study heterogeneity, heteroskedasticity, data dependencies, publication bias and truncated samples. The empirical results are consistent with publication bias, omitted variable bias in some studies, and lack of a genuine effect, especially for mass media. The paper also discusses ‘dissemination bias’ in the use of research results by primary investigators and health policy interest groups.     

POLITICAL BUDGET CYCLES: MANIPULATION BY LEADERS VERSUS MANIPULATION BY RESEARCHERS? EVIDENCE FROM A META‐REGRESSION ANALYSIS

Despite a long history of research on political budget cycles, their existence and magnitude are still in question. By conducting a systematic analysis of the existing literature, we intend to clarify the debate. Based on data collected from 1037 regressions in 46 studies, our meta‐analysis suggests that little, if any, systematic evidence can be found in the research record that national leaders do manipulate fiscal tools in order to be reelected. However, it is much more clear that researchers selectively report that national leaders do manipulate fiscal tools in order to be reelected. The publication selection bias highlighted has nonetheless been reduced during the past 25 years of research. We also show that the incumbents' strategies differ depending on which tools they use. Finally, the nature and quality of political institutions appear to be the factors which most affect the political budget cycles.     

Meta‐Analytic Research in Management: Contemporary Approaches,Unresolved Controversies,and Rising Standards

Early meta‐analyses in management research sought primarily to resolve seemingly conflicting findings by estimating a relationship’s population‐level effect size. Since then, management researchers have adopted increasingly sophisticated approaches that permit new theorizing, testing and comparing sophisticated models, and identifying boundary conditions. We summarize three of these approaches – i.e., qualitative meta‐analysis (QMA), meta‐analytic structural equation modeling (MASEM), and meta‐analytic regression analysis (MARA) – along with the special issue papers that adopt each approach. We conclude by raising three unresolved controversies that we believe deserve more attention and by offering our thoughts about how to maximize a meta‐analytic study’s chances for publication and impact.     

DOES INVESTOR PROTECTION INCREASE FOREIGN DIRECT INVESTMENT? A META‐ANALYSIS

We undertake a meta‐analysis of the effects of international investment agreements for the protection of foreign investors on foreign direct investment using 2107 estimates drawn from 74 studies. Our meta‐analysis finds robust evidence that effect of international investment agreements is so small as to be considered zero.  However, our results do not rule out the possibility that the effect of these agreements is, in fact, positive and that current research methods are insufficiently powerful or precise to identify the underlying genuine effect. FDI from developed countries appears to be more responsive to the existence of investment protection, and there is evidence of publication–selection bias in favour of studies that find a positive effect for investor protection.     

THE DEMAND FOR BEER,WINE AND SPIRITS: A SURVEY OF THE LITERATURE

Abstract The demand for alcohol literature is vast and much conflicting information about the nature of the demand for alcoholic beverages has been published. This paper presents a survey of the literature, and then uses the technique of meta‐regression analysis to establish insights into the nature of the demand for beer, wine and spirits. Unlike previous meta‐studies of the demand for alcoholic beverages this study adjusts for the precision of each elasticity estimate. The analysis presented suggests reported elasticity estimates will be influenced by such factors as estimation technique, data frequency and time period under consideration. With respect to time, the findings suggest that the demand for alcoholic beverages has become less inelastic since the mid‐1950s and that the income elasticity has been falling since the mid‐1960s. The analysis also found support for the idea that alcohol as a commodity group is a necessity, and that consumers respond to price discounting with inventory behaviour rather than true substitution behaviour. Little support is found for the idea that the demand for alcoholic beverages varies fundamentally across most countries, although wine may be an exception.     

ALCOHOL CONSUMPTION AND HUMAN CAPITAL: A RETROSPECTIVE STUDY OF THE LITERATURE

Abstract Grossman proposed an individual's health can be viewed as one aspect of their human capital. Following this line of thought a number of recent papers have reported a positive impact of alcohol consumption on earnings. The rationale for the existence of such a relationship is the positive impact of alcohol on physical and mental health. We conduct a meta‐analysis to determine whether such factors as: the estimation technique, the presence of ex‐drinkers in the sample, possible sample selection bias and publication bias may all contribute to these findings. An additional suggestion for the positive relationship between alcohol and wages is the presence of a common set of personality traits that determines drinking behaviour and also leads to higher earnings. We examine this relationship by reviewing the literature that investigates if the personality influencing aspects of alcohol consumption influences measures of human capital. We also survey the significant body of research that has examined how alcohol consumption has been found to influence educational outcomes and the work force participation of problem drinkers.     

TECHNOLOGICAL INNOVATION AND EMPLOYMENT IN DERIVED LABOUR DEMAND MODELS: A HIERARCHICAL META‐REGRESSION ANALYSIS

The effect of technological innovation on employment is of major concern for workers and their unions, policy makers and academic researchers. We meta‐analyse 570 estimates from 35 primary studies that estimate a derived labour demand model. We contribute to existing attempts at evidence synthesis by addressing the risks of selection bias and that of data dependence in observational studies. Our findings indicate that: (i) hierarchical meta‐regression models are sufficiently versatile for addressing both selection bias and data dependence in observational data; (ii) innovation's effect on employment is positive but small and highly heterogeneous; (iii) only a small part of residual heterogeneity is explained by moderating factors; (iv) selection bias tends to reflect preference for upholding prevalent hypotheses on the employment effects of process and product innovations; (v) country‐specific effect‐size estimates are related to labour market and product market regulation in six OECD countries in a U‐shaped fashion; and (vi) OLS estimates reflect upward bias whereas those based on time‐differenced or within estimators reflect a downward bias. Our findings point out to a range of data quality and modelling issues that should be addressed in future research.     

EDUCATION AND ENTREPRENEURSHIP SELECTION AND PERFORMANCE: A REVIEW OF THE EMPIRICAL LITERATURE

Abstract This paper provides a review of empirical studies into the impact of formal schooling on entrepreneurship selection and performance in industrial countries. We describe the main effects found in the literature, we explain the variance in results across almost a hundred studies, and we put the empirical results in the context of related economic theory and the much further developed literature in labor economics (studying the rate of return to education among wage employees). Five main conclusions result from this meta‐analysis. First, the impact of education on selection into entrepreneurship is insignificant. Second, the effect of education on performance is positive and significant. Third, the return to a marginal year of schooling is 6.1% for an entrepreneur. Fourth, the effect of education on earnings is smaller for entrepreneurs than for employees in Europe, but larger in the USA. Fifth, the returns to schooling in entrepreneurship are higher in the USA than in Europe, higher for females than for males, and lower for non‐whites or immigrants. In conclusion, we offer a number of suggestions to move the research frontier in this area of inquiry. The entrepreneurship literature on education can benefit from the technical sophistication used to estimate the returns to schooling for employees.     

WHICH FISCAL MULTIPLIERS ARE REGIME‐DEPENDENT? A META‐REGRESSION ANALYSIS

We analyse whether estimated multiplier effects are systematically higher if the economy suffers a downturn. For that purpose, we conduct a meta‐regression analysis on a unique data set of 98 empirical studies with more than 1800 observations on multiplier effects and control for regime‐dependence of the multiplier. We find that spending multipliers are much higher (by about 0.7–0.9 units) during a downturn. Tax multipliers are not sensitive to the economic regime, and generally lower than spending multipliers. Finally, for all spending categories other than government consumption, the multiplier significantly exceeds one during downturns.     

EXPLAINING DIFFERENCES IN EFFICIENCY: A META‐STUDY ON LOCAL GOVERNMENT LITERATURE

This paper reviews the literature on local government efficiency by meta‐reviewing 360 observations retrieved from 54 papers published from 1993 to 2016. The meta‐regression is based on a random‐effects model estimated with the two‐step random‐effects maximum likelihood (REML) technique proposed by Gallet and Doucouliagos. Results indicate that the study design matters when estimating a frontier in local government. We find that studies focusing on technical efficiency provide higher efficiency scores than works evaluating cost efficiency. The same applies when using panel data instead of cross‐section data. Interestingly, studies that use the Free Disposal Hull (FDH) approach yield, on average, higher efficiency scores than papers employing the data envelopment analysis (DEA) method, thereby suggesting that in this literature the convexity hypothesis of the production set is a matter. Finally, the efficiency of local government increases with the level of development of the analysed countries and is positively related to the national integrity of the legal system. The opposite holds when considering the corruption.     

THE EFFECTIVENESS OF ACTIVE LABOR MARKET POLICIES: A META‐ANALYSIS

This paper provides a meta‐analysis of microeconometric evaluation studies on the effectiveness of active labor market policies. The analysis is built upon a systematically assembled data set of causal impact estimates from 57 experimental and quasi‐experimental studies, providing 654 estimates published between January 1990 and December 2017. We distinguish between the short and longer term impacts in our analysis; at 6, 12, 24, and 36 months after program start. After correcting for publication bias and country‐specific macroeconomic characteristics, subsidized labor and public employment programs have negative short‐term impacts, which gradually turn positive in the longer run. Schemes with enhanced services including job‐search assistance and training programs do not have these negative short‐term effects, and stay positive from 6 until 36 months after program start.     

INFLATION AND CENTRAL BANK INDEPENDENCE: A META‐REGRESSION ANALYSIS

Abstract Using 59 studies, we perform a meta‐regression analysis of studies examining the relationship between inflation and central bank independence (CBI). The studies considered are very different with respect to the CBI indicator used, the sample of countries and time periods covered, model specification, estimators used and publication outlet. We conclude that there is a significant publication bias. However, we also find a significant genuine effect of CBI on inflation. Differences between studies are not caused by differences in CBI indicators used.     

THE FINANCIAL DEVELOPMENT AND GROWTH NEXUS: A META‐ANALYSIS

We conduct a meta‐analysis of the literature of financial development and economic growth. We cover a large number of empirical studies and estimations that have been published in journal articles. We measure the degree of heterogeneity and identify the causes of the observed differentiation. Among the most significant factors behind this heterogeneity is the choice of financial‐variable proxies, the kind of data used as well as whether a study takes into account the issue of endogeneity. Our results suggest that the empirical literature on the finance–growth nexus is not free from publication bias. Also, a genuine positive effect exists between financial development and economic growth.     

Combining Statistical Evidence

The combination of evidence from independent studies has a curious history. The origins reach back at least to the beginning of the 20th century. Since the mid‐1970s, meta‐analysis has become popular in several fields, among them medical statistics and the behavioural sciences. The most widely used procedures were perfected in early papers, and subsequently, a kind of groupthink has taken hold of meta‐analysis. This explains the need for a review in a statistics journal, destined for a statistical audience. Meta‐analysis is not a hot research topic among graduate students in statistics, and by writing this article, we hope to change this. We wish to point out the shortcomings of the mainstream view and exhibit some of the open problems that await the attention of statistical researchers. A host of competent reviews of meta‐analysis have been published, and several book‐length treatments are also available. We have listed many of these in the bibliography but cannot guarantee completeness.