Developing Classification Systems for the Measurement of Fidelity and Adaptation for an Evidence-Based Parenting Programs

Designers of Evidence-Based Programs emphasize maintaining fidelity or adherence to program curricula. Studies on the impact of fidelity have begun to accumulate; most have found a link between fidelity and outcomes. The issue of measurement has been given little attention. Many studies create a global fidelity score based on a limited sample of program activities. A more nuanced approach is to start with a program’s theoretical model and examine outcome-specific fidelity. For example, the New Beginnings Program (NBP), a group-format parenting program for families undergoing divorce, targets parenting skills (warmth, communication, minimizing exposure to conflict, and discipline) to protect children from divorce-related stress. We hypothesize fidelity of activities designed to foster each of these skills will be associated with increases in the respective parenting practices.

Facilitators may add their own material, beyond what is prescribed in the manual (i.e., adaptation), which has been linked with positive outcomes in prior studies. Using a modified version of a measure developed for teacher delivery of a school-based substance use program (Hansen), we code adaptations made by community-based clinicians delivering NBP. Specifically, we code whether adaptations reflect new information or teaching methods (e.g., lecture or group discussion), alignment with program theory (valence), perceived purpose, and whether the adaptation was in response to a stated participant concern. This trial is the first time NBP has been evaluated with large numbers of fathers and Latino parents. We expect that adaptation will be more common and more predictive of outcomes for members of these two populations.

In assessing fidelity, items were created from actions in the manual and rated on a 3pt scale (incomplete, partially-, and fully completed). For each of the 10 NBP sessions, trained coders observed one activity, which was chosen based on importance to NBP’s model. Inter-rater reliability for fidelity was .90 for cohort 1 (8 groups, 75 activities) and .91 for cohort 2 (12 groups, 111 activities). Cohorts 3 and 4 will be coded by May, 2014. The adaptation measure was adapted and refined based on qualitative observations in Cohorts 1 and 2 and will be coded by May, 2014. We will present the association between fidelity and outcomes, examining outcome-specific fidelity scores and their association with their respective outcomes, as listed above. We will also present a typology of adaptations and examine the hypothesis that adaptation valence (alignment with program theory) is predictive of outcomes. Finally, we will conduct multiple group analyses to examine the effect of parent gender and ethnicity.