WITHDRAWN: Exploring the Feasibility and Validity of a Function-Focused Approach Towards Implementation Fidelity in Complex Interventions

Introduction: To resolve the tension between preserving implementation fidelity and adapting complex interventions to specific contexts forms a key challenge to reach underserved populations with evidence-based prevention programs and conquer health inequity. To achieve this, some researchers argue that interventions should be defined by their functions: theorized change mechanisms occurring directly as a result of implementation, instead of their form: an exact set of implemented activities. To explore the feasibility of this idea in practice, we monitored the functions of Project-Based Learning (PBL) alongside the implementation of its form in Studio Schools. Studio Schools are innovative English secondary schools combining applied learning and coaching to promote student welfare and engagement with education, often in high-risk areas. Each school is encouraged to adapt the model to fit its local context, posing a particular challenge to implementation monitoring based on form across sites.

Methods: We specified the activities (form) and their assumed functions of PBL, and subsequently operationalized these as questionnaire items. Data on the implementation of PBL activities and the presence of functions were collected in four Studio Schools from October 2014 to January 2015. Data sources include qualitative and quantitative checklists completed by teachers and students, and quantitative checklists completed by researchers. All checklists comprise of scales structured per PBL component, with separate subscales for activities and functions. Respondents were asked to rate if they agree that an item (activity or function) is implemented or otherwise present on a 4-point scale ranging from ‘completely disagree’ to ‘completely agree’.

Results: Factor analysis employed on student-reported data suggested that form- and function-focused fidelity can be measured simultaneously with validity, although limitations apply. Inter-rater reliability analysis of responses by participants and researchers (expressed as ICCs) resulted in differences in interpretation between different observers. This demonstrates the importance of actively involving target communities as partners in fidelity assessments.

Conclusion: This study demonstrates that the specification and subsequent monitoring of program functions is feasible. This approach may allow for increased flexibility in implementation across sites without necessarily compromising implementation fidelity and its monitoring in prevention science. Overcoming this challenge is paramount building an evidence base for programs and theories that can be adapted for specific and/or marginalized communities.