Holding back from theory: limits and methodological alternatives of randomized field experiments in development economics

ABSTRACT

In this paper, we critically and constructively examine the methodology of evidence-based development economics, which deploys randomized field experiments (RFEs) as its main tool. We describe the context in which this movement started, and illustrate in detail how RFEs are designed and implemented in practice, drawing on a series of experiments by Pascaline Dupas and her colleagues on the use of bednets, saving and governance in Kenya. We show that this line of experiments have evolved to address the limitation of obtaining policy-relevant insights from RFEs alone, characterized as their lack of external validity in the literature. After examining the two prominent responses by leading figures of evidence-based development economics, namely machine learning and structured speculation, we propose an alternative methodological strategy that incorporates two sub-fields, namely experimental economics and behavioral economics, to complement RFEs in investigating the data-generating process underlying the treatment effects of RFEs. This strategy highlights promising methodological developments in RFEs neither captured by the two proposals nor recognized by methodologists, and also guides how to combine different sub-fields of economics.