A Longitudinal Investigation of the Relations Among Dimensions of Implementation, Peer Victimization and Prosocial Behaviors

Introduction: That program implementation affects program outcomes is not controversial. However, as argued by Durlak and DuPre (2008), the assessment of implementation and its impact on program effectiveness is complicated by the multidimensional nature of implementation. Implementation has often been assessed with measures of whether the program is implemented as intended (adherence/fidelity), or dosage (the frequency of use of program components). Less attention has been paid to other implementation aspects such as quality (how well different program components have been conducted), or participant responsiveness (participants involvement and interest in the program). Moreover, few studies have evaluated how aspects of implementation change over time. Using data from the WITS bullying prevention programs (www.witsprogram.ca) designed for children in grades 1 to 6, this study examines: (1) changes in and relations among four aspects of implementation across 2 years (4 waves of data) including adherence, dosage, quality and participant responsiveness, and (2) investigates which aspects of implementation have an impact on key program objectives: reducing peer victimization and increasing pro-social behavior among elementary school children over time.

Method: Data for this study were drawn from a recent national randomized control trial evaluating the WITS Programs. Data were collected 4 times between fall 2011 and spring 2013. The current study uses only program school data which included 1301 children (M=8.9, SD=5.2; 52% female), as well as their teachers and parents.

Results: Levels of adherence and dosage were highest at T1 and T2 and then stabilized or decline at T3 and T4. In contrast, quality increased over time and participant responsiveness was initially high and remained stable over time. Cross-sectional analysis showed that quality of program implementation was significantly associated with lower levels of both relational (β=-10, p<0.01) and physical victimization at T4 only (β=-.15, p<0.05). Higher levels of adherence (β_range=.25 - .46, p<0.05), quality (β_range=.10 - .13, p<0.05) and participant responsiveness (β_range=.61 - .48, p<0.05) was associated with higher levels of prosocial behavior at each time point. Cross-time analysis will also be assessed.

Discussion: The findings highlight the importance of considering the unique effects of different aspects on implementation on program outcomes over time. Quality and participant responsive were stronger indicators of program effectiveness than adherence or dosage alone. Further, the strongest effects were found 2 years after program implementation, suggesting the timing of implementation assessment may be important.