An Integrative Mixed Methods Analysis of Evidence-Based Intervention Sustainability and Adaptation

Introduction: Prevention Science has recently emphasized the broad dissemination and implementation of efficacious evidence-based interventions (EBIs) (Spoth et al., 2013). By contrast, the extensive diversity existing in populations and communities nationally has accentuated the fidelity-adaptation dilemma (Barrera et al., 2013; Hanson et al., 2013). Adaptive changes in an original EBI are often necessary, yet must also maintain fidelity to the EBI’s scientifically-established core contents. The present study conducts a deep-structure analysis of: (a) factors affecting EBI sustainability, and (b) teacher-initiated adaptations, to (c) thus develop Principles and Guidelines for efficacious adaptations.

Methods: A mixed-methods (QUAL+QUAN) research design is used to conduct key informant interviews with principals and teachers from rural schools in Pennsylvania and Ohio. These schools participated previously in a randomized controlled trial of the keepin’ it REAL EBI. Our “Principal Interview & Questionnaire” consists of a qualitative interview and a quantitative questionnaire, that focuses on the construct of EBI Sustainability. It examines: (a) the process of adopting and sustaining this EBI; and (b) as EBI funding ends, strategies and actions for sustaining this EBI. Using this QUAL+QUAN structure, the “Teacher Interview & Questionnaire” focuses on Theoretical Congruence and examines: (a) teacher-initiated adaptations and their rationale, and (b) teacher-generated Principles and Guidelines of effective EBI adaptation. Teachers will also be re-interviewed in a “member check” analysis for validating initial Principals and Guidelines. Thematic analyses are being conducted on focus question narratives using the Integrative Mixed Methods (IMM) methodology (Castro et al., 2010). Each teacher-initiated adaptation will be rated for theoretical congruence by two independent research assistants. Then in a roundtable analysis, developers of this EBI will provide additional congruence ratings to cross-check ratings and improve the validity of these ratings.

Results: Initial evidence from principals’ narratives shows that sustainability is not determined solely by the interests of principals and schools. Externally-imposed educational mandates for accountability in teacher and school performance are constraining the full implementation of the original EBI.

Conclusions: Beyond teachers’ local adaptations, complex systemic community-based factors are influencing EBI adaptations and sustainability. Implications of these systemic influences on EBI adaptation, sustainability, and future adaptive designs will be discussed.