Incorporating Intervention Dosage Into Analysis of Family Interventions

Introduction:  Family-focused interventions share the common problem of less than full recruitment of families and less than full participation of those agreeing to participate.  Because of this, there is reason for concern that intent-to-treat (ITT) analyses might incorrectly estimate effects for those who were exposed to the intervention.  This paper reports two methods for incorporating actual dosage for participants and predicted dosage for controls into the analysis. 

 

Method and Results:

Study 1 used weighting by predicted dosage to evaluate proximal effects of the family intervention of the Multisite Violence Prevention Project, where 45% of eligible families participated in one or more sessions.  Treating dosage among control subjects as missing data, and using imputation, we weighted cases by actual dosage for the treatment group and predicted dosage for the control group in the analysis.   The resulting models rendered some effects significant that had been non-significant in ITT analyses, and other effects non-significant that had been significant in ITT analyses. 

Study 2 used propensity scores to evaluate the effectiveness trial of the SAFEChildren intervention, in which 52% of eligible families participated in one or more sessions.  There were four clear groups of participants:  1) no consent, 2) no participation, 3) low participation, and 4) high participation.  We estimated propensity scores for each group using baseline measures from four domains: 1) primary caregiver characteristics, 2) student characteristics, 3) family functioning, and 4) neighborhood characteristics.  A multinomial model of participation group yielded 86.3% predictive accuracy within the treatment group.  Analyses using propensity score matching found a positive effect on social skills that had been marginal in ITT analyses, and a significant negative effect on family cohesion that had not appeared in ITT analyses.

 

Conclusion:  Incorporating actual and predicted intervention participation into analyses of interventions with large participation variance retains many of the strengths of random assignment but provides more sensitive and specific effect estimates.  When true intervention effects are present, incorporating dosage should return more robust effects than an ITT analysis that includes those assigned but not receiving intervention.  When true intervention effects are absent, incorporating dosage can identify spurious ITT effects.  Without the ITT analysis, dose-weighted analysis may return significant effects because of nonrandom factors such as selection bias.  Without incorporating dosage, an ITT analysis may incorrectly conclude that an intervention is not effective.