Examining the Impact of an Online Training On Role Play Implementation in an Evidence-Based STD, HIV and Pregnancy Prevention Program

Introduction

The use of interactive activities (e.g., role play) is one of the key characteristics of effective programs (Kirby, 2007). Through a NIH-funded SBIR, we evaluated an online training program designed to improve the implementation fidelity of teachers implementing the Reducing the Risk (RTR) program - a pregnancy, STD and HIV prevention evidence-based curriculum. Our current analysis examines the impact of the online training on a list of 42 core activities that were a specific focus of the online program to improve role play, and for which we would hope to see significant differences.

Methods

The study was a two-group randomized-control trial of a national cohort of teachers implementing RTR.  The control group was provided with the RTR curriculum guide, while the treatment group was provided with the RTR curriculum guide as well as access to the RTRworks! online training.

Educators completed a pre-survey prior to implementation and a post-survey after implementation. They also completed implementation logs for all 16 lessons. Classroom observations, audio recordings and interviews were additional methods used to collect data. Finally, ‘dosage’ on use of the online training was obtained from website tracking metrics.

Results

An average fidelity percent was computed for each participant. The online training group reported significantly higher implementation fidelity to all activities than the standard group (p=0.05). Implementation fidelity for the targeted core activities was also significantly higher for the online training group (p=0.02).

At posttest, significantly more educators in the online training group expressed that their skill level in facilitating role plays had improved (p = 0.00). Dosage analyses performed for the online training group yielded statistically significant outcomes for ‘knowledge of role play management,’ and ‘attitudes towards role play.’ All 10 interview participants from the online training group found the training to be valuable, and said it helped with their implementation of the role plays. Much of the support that participants in the control group requested were tools that were offered in the online training.

Conclusion

An online training curriculum is a viable and cost-effective means of increasing educators’ sense of preparedness to implement role play activities. Our survey found that 30% of the teachers “rarely or never” used role play in their current role as a teacher. This analysis will contribute to an overall understanding of teachers’ use of online training programs in general, and offer a closer look at how teachers utilized online tools to improve their role play facilitation skills.