More Than Fidelity: Why Context Matters for Youth Development Program Implementation

Introduction: Positive youth development programs are designed to promote healthier decision-making among youth. A core component of such programs involves supportive adults mentoring youth (Larson, 2006), although research has been lacking about best practices for integrating mentoring programs within a larger system (DuBois & Rhodes, 2006), such as schools. There is also an increasing focus on adherence to fidelity when scaling up and replicating large prevention programs, but such studies often do not take into account the contextual factors that may influence implementation.

Methods: This paper presents data from an ongoing evaluation study of the Teen Pregnancy Prevention Initiative (TPPI) in a large urban school district. The analysis is based on observation data from 91 randomly-sampled sessions of the Wyman Teen Outreach Program® (TOP), implemented across 16 schools during the 2012-2013 school year. A qualitative, grounded theory approach was used for the contextual analysis using Atlas.ti. A total of 49 distinct codes emerged from the data, which were then collapsed into broader categories comprised of classroom level factors (positive and negative teacher behaviors), school level factors (sexual health policies and scheduling), and district level factors (programmatic decisions and facilitator attrition).

Results: Using evidence from field notes, we found that teachers are a critical contextual factor to consider at the classroom level. Many teachers tended to be disengaged during sessions but might respond to a minor behavioral issue. While some did make positive contributions and promoted TOP (n=18), of concern were instances when teachers became a negative presence in the classroom (n=26). Some would undermine the safe space facilitators attempted to create by making sarcastic comments towards students, while others disrupted the flow of the session. At the school level, it appeared that variable policies regarding sexual health education and school schedules appeared to be a critical factor in the consistency of TOP implementation. Finally, at the district level, we found there was an emphasis on adherence to fidelity monitoring, particularly related to curriculum delivery. This limited facilitator engagement with students and schools and in turn detracted from the ability to build relationships important to youth development.

Conclusions: This study sheds light on contextual factors influencing program implementation that are not readily apparent from survey analyses or monitoring fidelity. Sharing findings with practitioners in real-time allowed us to target recommendations for modifying practices, such as educating teachers about youth development and allowing TOP facilitators to spend more time in schools on non-implementation days.