Using Complier Average Causal Effect Estimation to Determine Student Outcomes of the Good Behavior Game Preventive Intervention

Introduction: An alternative to traditional intent to treat analyses (ITT) is Complier Average Causal Effect (CACE; Jo, 2002) analysis, which is a causal inference approach that is used to account for levels of implementation compliance. CACE has been used to estimate treatment effects accounting for compliance; however, there has been less focus on classroom-based preventive interventions implemented by teachers. This study used CACE to estimate the impacts of an integrated version of the PAX Good Behavior Game and Promoting Alternative Thinking Strategies (i.e., PATHS to PAX), relative to PAX GBG-only and a control, on student outcomes, taking into consideration teacher compliance (i.e., implementation fidelity).

Methods: The sample included 350 K-5 teachers across 27 schools. The student sample consisted of 1526 children (60.6% male) who were rated as displaying higher levels of teacher-reported aggression (i.e., top 33 percentile). The majority of students in the schools were African American (93.7%) and received free and reduced meals (92.5%). The following 6 student outcomes were assessed using a teacher-report measure administered as a fall baseline and follow-up at the end of the school year: academic engagement, social competence, peer relations, emotion regulation, hyperactivity, and aggression.

Results: After performing the traditional ITT analyses of the effects of the interventions on the 6 outcomes, CACE models were estimated separately for each of the treatment conditions relative to control (i.e., PATHS to PAX integrated v. control and PAX GBG-only vs. control). More specifically, linear regression models were fit for each of the outcomes, controlling for baseline scores, along with other student demographics (i.e., student gender, referral for special education, race). CACE was defined as the effect of intervention assignment for compliers on post-intervention scores for each of the 6 student outcomes. We used two different cut points (50^th and 75^th percentile) for compliance as a sensitivity analysis. The findings indicated stronger effects on students for the combined PATHS to PAX program, relative to both controls and PAX GBG-only, in both the ITT and CACE analyses. The effects of the program were stronger when compliance with program implementation expectations was taken into account.

Discussion: The ITT and CACE results echo similar effects of the integrated PATHS to PAX intervention with regard to student outcomes, as were previously documented on teacher outcomes. Furthermore, the PATHS to PAX effects were particularly salient when compliance was taken into consideration. Implications for future research will be discussed.