Interrelation's Among the Dimensions of Implementation over Time: Enhancing the Utilization of Evidence Based Prevention Programs

Introduction: Inadequate monitoring of implementation can create uncertainties about whether programs were delivered as intended. Moreover, short-term evaluations may not allow sufficient time for uptake and infusion of the program in concentrations sufficient to effect change. Durlak and DuPre (2008) identify 8 dimensions of program implementation including adherence to program curriculum, dosage (how much), quality (how well) and participant responsiveness (program engagement). Berkel et al. (2011) proposed a theoretical model of the interdependency among dimensions of implementation and called for longitudinal research to disentangle the bidirectional associations over time. Building on this theoretical model, we use data from a longitudinal randomized cluster evaluation of the WITS programs (www.witsprogram.ca) to examine concurrent and prospective relations among three dimensions of implementation (adherence, quality, and participant responsiveness), across 2 years (4 waves). Specifically, we test (1) whether adherence and quality predict participant responsiveness within and across time, and (2) whether participant responsiveness predicts adherence and quality.

Method: Data are from a national randomized cluster trial evaluating the WITS Programs designed for the prevention of bullying and peer victimization among children in grades 1-6. Data were collected 4 times between fall 2011 and spring 2013. The current study uses only program schools (1329 children; M=8.2 years, SD=1.30; 52% female). Each implementation measure had five items from multiple-informants (child, parents, and teachers). Path analysis assessed the associations among adherence (i.e., Read 3 or more books in the semester (teacher report)), quality (i.e. I brought home a WITS gift (child report)), and participant responsiveness (i.e., I used my WITS to deal with bullying).

Results: Autoregressive paths showed small to moderate stability in each dimension over time. Correlations among dimensions were stronger within time than across time and were strongest at T5 (range r = .37 to .58). Higher levels of adherence and quality were associated with higher levels of participant responsiveness at each time. Levels of participant responsiveness predicted subsequent levels of adherence and quality 6 months later (best fitting model: χ²(39) = 217.01***, RMSEA = .06, CFI  = .93). Levels of adherence and quality did not predict subsequent participant responsiveness across time.

Discussion: The findings highlight the need for continued support for program adherence and quality within and across school years in order to sustain participant responsiveness. Moreover, the findings show that high levels of program engagement by participants act to reinforce teacher’s program adherence and quality 6 months later.