Assessing Implementation Delivery of Classroom-Based Prevention: Years 1 and 2 Results of an Efficacy Trial of Best in CLASS

Introduction: Many preschool children exhibit problem behaviors (behavioral, emotional, social) and collateral pre-academic deficits that place them at risk for emotional and behavioral disorders. Evidence-based programs comprised of evidence-based instructional practices delivered by teachers in classrooms have demonstrated positive effects for young children with problem behavior; however, efforts to implement and evaluate these programs across a variety of early childhood programs face barriers. BEST in CLASS was designed to comprise evidence-based instructional practices to improve positive teacher-child interactions and reduce instances of challenging behavior, with an eye toward overcoming implementation challenges via practice-based coaching.

Theoretical models of implementation have been offered to guide implementation efforts in school-based prevention. Berkel et al. (2011) provided a model that posits that implementation is moderated by competence of delivery and child responsiveness, and the effect of quality on outcomes is mediated by child responsiveness. BEST in CLASS measures three dimensions of program implementation, via direct observations, that may ultimately help explain program outcomes: adherence, competence of delivery, and child responsiveness. This paper describes how these dimensions of implementation of the BEST in CLASS program change over time as a function of the practice-based coaching component of the model.

Methods: Trained observers measured treatment integrity on three dimensions (Adherence, Child responsiveness and Competence; 7-point scale) using the BEST in CLASS Adherence and Competence Scale. One thousand seven hundred and fifty eight observations of 48 teachers in BEST in CLASS (n = 116 children) and 50 comparison (n = 103 children) classrooms were conducted over approximately five months during years 1 and 2 of a 4-year RCT examining the efficacy of BEST in CLASS.

Results: Adherence subscale scores varied across the treatment phase by group, and significant differences, all favoring the BEST in CLASS group, were noted at both post-treatment and maintenance. Similarly, Competence subscale scores varied across the treatment phase by group, and significant differences, all favoring the BEST in CLASS group, were noted at both post-treatment and maintenance. No differences between groups were noted for child responsiveness; component analyses indicated that teachers increased and maintained their use (adherence and competence) of evidence-based instructional practices following practice-based coaching.

Conclusions: These findings suggest that practice-based coaching is a promising tool for increasing teachers’ adherence to treatment delivery as well as the competence with which they implement treatment components.