Sustaining Fidelity in Wide-Scale International Implementation

To improve the reach of evidence based interventions, we require science-based implementation systems and a way to ensure that programs are delivered with sustained model fidelity. Direct observation of intervention sessions can evaluate potential model drift. Technological advances can provide purveyors access to video recordings of intervention delivered during training and afterwards across distances and implementation sites. The use of such direct observation can protect against program drift.

The Oregon Model of Parent Management Training (PMTO) is an evidence based program that fully transfers the intervention to adopting communities. The PMTO team trains a community progenitor group who are then trained to carry the program forward to future generations. This approach begs the question: can method fidelity be sustained after transfer? We address this question with observational data from several large-scale implementations.

Method: Data are based on the Fidelity of Implementation (FIMP) measure, which scores video recordings of intervention sessions. The measure has predictive validity, showing that high FIMP scores obtained during intervention predict pre/post changes in observations of parent/child interaction and pre/post parent report of child behavior problems. In the present study, fidelity scores were collected by 4 nationwide implementation sites: Norway, Iceland, Denmark, and the Netherlands. Each site provided FIMP data at certification of trainees. Generation 1 (G1) practitioners were trained by the purveyor. Subsequent generations were trained by the community. Earlier studies showed FIMP certification scores for G2 practitioners to decline significantly from G1 scores. However by G3, FIMP scores returned to the levels obtained by G1 and sustained to G4. In this study, we examine FIMP scores from 4 implementation sites assessed as far as the seventh generation. We hypothesize a drop at G2 in each site with a return to G1 levels by G3 and thereafter.

Results: As hypothesized, the observed fidelity scores showed the decline from G1 trainees at G2. However, in all cases the fidelity scores returned to the G1 levels within one or two generations and sustained thereafter.

Conclusion: Given that training and coaching are not supervised by the program purveyor after transfer to the community, evaluating sustained fidelity is essential to ensure that drift has not occurred. This study suggests that the full transfer method of implementation can be carried out with sustained fidelity using direct observation.