Session: Advancing Quasi-Experimental Methods: Comparative Regression Discontinuity Design (Society for Prevention Research 23rd Annual Meeting)

A big challenge societies face is to demonstrate the effectiveness of social policies and programs. Randomized controlled trials (RCT) are treated as the gold standard to justify causal conclusions. Although the appeal of RCTs is apparent, it is not typically feasible to randomly assign laws, policies and programs to communities. The next best alternative is the regression discontinuity design (RDD), and it is accepted into many compendia of effective practices such as the What Works Clearinghouse in Education and Blueprints in the prevention sciences. However, RDD has important limitations: first, the functional form of the relation between the assignment variable and the outcome variable has to be correctly. Secondly, RDD has lower statistical power due to the correlation between the assignment variable and the treatment. A third limitation is that RDD has limited generalized causal inference away from the cutoff score. The aim of this symposium is to propose and show the performance of a new variant of RDD, the comparative regression discontinuity (CRD) design. It adds to the basic RDD comparison functions coming from either (1) pretest scores, or (2) a matched non-equivalent comparison group. The symposium shows how either supplement can help overcome the three limitations of basic RDD listed above. To do so, the three studies use a novel methodology, within-study-comparisons, to assess the performance of CRD compared to the results of an RCT that serve as the causal benchmark. The first paper examines the consequences of adding a pretest measure of the study outcome using a randomized experiment where families with a disabled person are randomly assigned to sums of money. A synthetic RDD and a CRD are constructed from these experimental data. The study explores the causal estimates at the cutoff and above the cutoff, and shows that all three limitations of basic RDD are mitigated. The second paper examines the effect of using a non-equivalent comparison group or a proxy-pretest as the comparison function. The study explores the accuracy of estimates when the functional form assumption fails using data from a highly structured four-arm experimental study. The third paper conducts six within study comparisons based on three outcomes and two assignment variables using a national program evaluation data set. The study investigates which is better for reducing bias and increasing statistical precision – using pretest or a non-equivalent control group as the comparison function. Taken together, the authors claim that both prospective and archival RDD studies make use of a comparison regression function widely possible and so recommend using CRD instead of RDD whenever possible.