Calculating long-run impacts of social programs with staggered implementation

When measuring the causal impacts of policy interventions, evaluators often contrast samples with and without the intervention tracked longitudinally. This article considers what happens when the without-intervention portion of the sample begins to be treated with the intervention part way through the follow-up period. The authors focus on randomized control trials, whose sponsors increasingly guarantee all participants in the study the intervention—the control members with a lag—while noting that the problem and their proposed solution apply to non-experimental impact evaluations as well. Once the control group (or other counterfactual group) receives the intervention, the treatment-control comparison becomes a measure of the difference in impacts between interventions of different durations—longer in the treatment group than the control group—rather than a measure of the total impact of the longer duration compared to no intervention. To measure this total impact—which tends to matter more to policy than differential or short-term impacts—the authors propose to sum the available experimental impact estimates from consecutive time periods, thus combining an experimental measure of the impact of the short duration intervention with an incremental measure of the added impact of the longer duration. The statistical properties of this estimator are explored, as are the conditions under which it provides an unbiased estimate of the total long-term effects of treatment.