Session: Leadership, Dosage, and Outcomes: Lessons from the Toolbox Implementation Research Project (Society for Prevention Research 25th Annual Meeting)

Nearly 20% of youth in the US have a mental, emotional, or behavioral disorder (O’Connell et al., 2009). Many of these problems are preventable. School-based prevention (often, Social Emotional Learning; SEL) programs have been shown to prevent anxiety, conduct problems, delinquency, drug use, and truancy, while promoting emotional regulation, prosocial skills, and academic achievement (e.g., Abbott et al., 1998; Domitrovich et al, 2007; Durlak et al., 2011; Espelage, et al., 2015; Flay & Allred, 2003; Greenberg et al., 2003). Research indicates that teachers view SEL as essential for students and are generally eager to implement SEL programs (Shapiro et al., 2016). Yet many SEL programs are highly structured (a preference of only a minority of teachers) and do not necessarily fit into routine teaching practice. Thus, effective programs are not having a broad impact.

TOOLBOX (Collin, 2015) is a SEL program that is flexible by design. Administrators can adopt the program in a way that emphasizes either lesson-based or strategies-based delivery. Teachers are given options within these frameworks for how to dispense the (theorized) active ingredients. This relatively unique approach may explain why TOOLBOX has quickly been adopted by over 40 school districts in Northern California. Although TOOLBOX is popular, it has not yet been tested scientifically. Much of the work involved in testing such a program requires understanding how varying implementation contexts and behaviors relate to youth outcomes.

The TOOLBOX Implementation Research Project (TIRP) used a quasi-experimental design to identify the essential implementation practices associated with intervention success. The first paper uses the Implementation Leadership Scale (Aarons et al., 2014), previously used in clinical settings, and finds acceptable, although somewhat lower, alphas and ICCs. Implementation leadership was moderate-to-good in October and declined throughout the school year. The second paper studies dosage under two different implementation directives. Despite differences in the emphasis of lessons relative to strategies, dosage rates were similar on lesson-based and strategy-based indicators. Lesson-based dosage increased over time, while strategy-based dosage decreased over time. The third paper studies the relationship between implementation leadership, dosage, and youth outcomes. Youth at TOOLBOX schools had better outcomes than youth at comparison schools, regardless of whether they were directed to emphasize lessons or strategies. Findings suggest that implementation leadership and dosage reported by teachers both related to youth outcomes. The symposium will conclude with comments and discussion facilitation by an SEL senior scientist.