Strengthening the Regression Discontinuity Design

Introduction: The basic regression discontinuity design (RD) suffers from three major limitations relative to the randomized clinical trial (RCT). It is more dependent on functional form assumptions; its power is lower by a factor of at least 2.75, and its causal inference is less general, being limited to cases immediately around the cutoff score determining treatment assignment. It would be nice to mitigate all three of these limitations with a simple design element that could be added to the basic RD. The current paper examines the assumptions, the causal bias and the precision of the comparative regression discontinuity design (CRD) relative to the assumptions, bias and precision in basic RD and the RCT. Specifically, the paper examines the consequences of adding a pretest measure of the study outcome as the no-treatment regression function that the CRD needs.  

Method: In this design, the RCT serves as the causal benchmark. It involved randomly assigning families with a disabled person to the exact sum of money they could spend either under their own control or under administrator control. All the money was meant to support improving the life of the disabled family member. The outcome examined is the fraction of the allotted money actually spent to support the disabled person. The RDD and CRD were constructed from the RCT by omitting the treated cases on the untreated side of the assignment variable and the untreated cases on the treated side.  

Results: Results show that (a) the untreated functional form in the RD is similar to the untreated forms in the CRD, thus supporting the inference that the regression function in the untreated part of the basic RD would have continued as it was if the units on the other side of the cutoff had not received treatment. (b) The obtained standard errors of causal estimates were close in the CRD and experiment but were distant in the basic RD — thus supporting the case that precision improved by virtue of adding the no-treatment pretest regression function. (c) Causal estimates at the cutoff were similar in the RD, CRD and experiment, thus demonstrating the usual and expected unbiased nature of causal inference at the cutoff. And (d) most importantly, causal estimates away from the cutoff were similar in the CRD and RCT, thus demonstrating that under the conditions studied causal inference in the RD was not limited to around the cutoff score.  

Conclusion: In sum, all three limitations of basic RD were demonstrably mitigated by adding to the basic RD a no-treatment regression function derived from pretest measures of the study outcome.