Improving Implementation Fidelity through Evidence-Informed Implementation Assessment and Feedback: T F C O As a Case Example

Introduction

Although there have been multiple examples of system attempts to scale-up evidence-based practices (EBPs) to treat high-risk populations, these efforts often are met with challenges that hinder the development of a sustainable program. This presentation will focus on development of an implementation strategy to assist organizations in obtaining implementation fidelity to increase the potential of successful adoption and sustainability of EBPs. The Stages of Implementation Completion (SIC) was developed as an observational measure of implementation process and milestones. The SIC has repeatedly demonstrated success in predicting implementation outcomes across a range of EBPs. It has been programmed as a web-based platform, with regular feedback available regarding adopting site’s progress toward successful implementation.

Treatment Foster Care Oregon (TFCO) is an EBP alternative for youth referred to out-of-home care. Standard implementation of TFCO occurs with the assistance of a website (i.e., FOCUS) to provide sites with information regarding intervention fidelity including site performance (e.g., recruitment and retention, caseload size, treatment delivery) and client outcomes.

A collaboration between the TFCO developer, purveyor, and SIC teams to integrate the SIC and TFCO fidelity platforms will be described. The goal is to provide newly adopting sites with comprehensive feedback regarding both implementation and treatment fidelity during weekly coaching calls, to help improve the potential for development of a sustainable program.

Methods

The SIC-Coaching technology platform leverages data from 376 previous TFCO implementation attempts, collected as part of a study to evaluate the reliability and validity of the SIC. From these data, performance metrics were derived to provide benchmarks for implementation behavior that predict success. Modeling was conducted to create an interactive website that provides prediction of success of an individual site compared to previous successful sites.

The SIC website is integrated with FOCUS to produce a comprehensive set of empirically based feedback regarding implementation and treatment fidelity of TFCO. This synthesis of data is used to inform weekly coaching between purveyors and adopting organizations.

Results

TFCO purveyors reported a strong desire to integrate organizational implementation fidelity into their process. Preliminary outcomes from initial use of the SIC-Coaching model with newly adopting sites, including user friendliness, ease of use during weekly coaching, and progress toward successful implementation will be described.

Conclusions

Use of technology to provide implementation and treatment fidelity feedback has the potential to inform successful implementation efforts. Lessons from this case study can help to inform the generalization of the strategy across a range of EBPs.