Session: Observation-Based Methods for Measuring EBP and Implementation Fidelity Using Technology-Assisted Platforms (Society for Prevention Research 25th Annual Meeting)

Voltage drops notwithstanding, it is necessary to deliver Evidence-Based models with high fidelity to yield positive outcomes in “real world” settings that are comparable to those achieved in RCTs. The same is true for implementation methods; how well an EBP is implemented from pre-implementation planning through sustainability is critically important to the long term survival of EBP models in routine care settings. This symposium will describe three observation-based fidelity systems that provide rapid feedback to adopting providers and organizations with the assistance of technology. Two were designed to measure fidelity of specific EBPs (Parent Management Training Oregon and KEEP) whereas the third measures the fidelity of implementation (Stages of Implementation Completion). This symposium will demonstrate these respective observation-based fidelity systems, and describe improvements in outcomes with the use of rapid feedback systems.

The first paper will describe the Fidelity of Implementation (FIMP) measure and its associated web-based platform. The FIMP is a method for scoring observations of Parent Management Training Oregon, and can be provided remotely through the use of video uploads to a secure web portal. This system allows for the efficient transfer of fidelity monitoring from the purveyor to the site, as part of a train-the-trainer type model. Fidelity outcomes across generations of trainees will be described.

The second paper will describe the FIDelity Observation (FIDO) web-based platform for monitoring of fidelity and provision of feedback for the KEEP model. Use of technology based monitoring allows for remote observation and rapid feedback, thereby increasing the capacity for scale-up across multiple regions within a system. Outcomes related to the use of FIDO to help support a system-wide adoption of KEEP will be described.

The third paper will focuses on supporting organizations attempting to adopt a new EBP. The Stages of Implementation Completion (SIC) web-based platform allows for integration with other web-based fidelity systems, to provide sites with comprehensive feedback regarding their combined implementation and treatment fidelity outcomes. This paper will describe ongoing efforts to support successful implementation through the use of easily accessible, rapid organizational level feedback. Treatment Foster Care Oregon (TFCO) is used as an example.

Discussion will include a focus on the advantages of observation-based measures relative to self- report, and how technology can assist new programs to achieve sustainable outcomes. The process by which these fidelity systems, developed for research, were adapted to fit the pragmatic demands in usual care settings will be discussed.