Advances in Research on Participant Attrition from Prevention Intervention Studies

This paper describes a promising approach to assessing and reducing participant attrition, a methodological challenge common in prevention research. Attrition undermines the internal validity of studies evaluating the efficacy/effectiveness of preventive interventions. Reducing the impact of attrition is key to these essential evaluations, and methods for assessing the degree of attrition bias are sorely needed.

We describe a paradigm for studying attrition. The paradigm, employed by Collins et al. (2001), involves generating missing data taking several factors into account, and assessing bias using standardized bias, a measure that quantifies bias as the percent of a standard error. We extend the paradigm by varying additional attrition-relevant factors, and by making use of relative bias (percent of the true parameter value) as well as standardized bias to assess attrition bias.

Graham (2012) suggested a taxonomy of eight cases of attrition that consider all possible combinations of the treatment (vs. control: T), the dependent variable (Y), and the interaction between them (TY) as causes of missingness. Attrition is not a problem with two of these cases (missingness is MCAR or MAR). Previous research (Collins et al, 2001; Graham et al., 2008; Graham, 2012) has shown minimal attrition bias in a third case (Y only as the cause of missingness).

We have also extended the paradigm by studying the remaining five cases, especially the case in which all three factors (T, Y, and TY) are causes of attrition. Conventional wisdom is that we cannot know from empirical data whether missingness is MAR or NMAR. However, in longitudinal prevention research, much attrition-relevant knowledge is possible.

Using the Colby, Hecht et al. (2013) study as an example, we describe the process of gleaning attrition-relevant information from empirical research. This process is not unlike gleaning information for studying statistical power. We describe a utility that uses analytic and Monte Carlo simulation methods (along with the empirical prevention data) for studying the impact of attrition bias in any of the eight cases of the attrition taxonomy. Using the utility, we show the extent of attrition bias in the actual empirical situation. We also show the bias when the level of attrition is doubled, and with a 10 fold increase in the difference in attrition in treatment and control groups.

Graham (2012) also suggested collecting attrition-relevant data (mobility) for reducing the impact of attrition. We show the benefits of adding these measures in our analysis. We show that the benefits are seen under all conditions, but are most dramatic where we need them most – where the bias due to attrition would otherwise be large.