Applying the Multiphase Optimization Strategy (MOST) to Engineer an Effective Substance Use Prevention Program for College Student-Athletes

Introduction: College student-athletes are at increased risk of substance use compared to non-athletes. Currently there are no evidence-based programs that address their unique patterns and motivations for use. In response to this void, we developed myPlaybook, an online substance use intervention tailored to college student-athletes. After initial development and evaluation of myPlaybook, we adopted an innovative, engineering-based methodology, the Multiphase Optimization Strategy (MOST), to optimize its impact. As part of MOST, we tested whether an initial set of intervention components (here, different myPlaybook lessons) met specific optimization criteria in a series of component selection experiments. Results from the first iteration indicated that myPlaybook did not have the intended effect, so we substantially revised myPlaybook. In this paper, we present results from our second iteration, which was conducted with the revised myPlaybook lessons.

Method: We used a multilevel fractional factorial design, with N = 3,284 freshmen college student-athletes nested within N = 47 schools. Each school was assigned to one of eight conditions. Participants completed an introductory lesson and some combination of three other lessons (i.e., social norms, expectancies, and harm prevention), depending on the condition to which their school was assigned. They also completed a web-based pretest survey, an immediate posttest survey and a 30-day follow-up survey. As part of all 3 surveys, students answered questions about behavioral intentions (e.g., intentions to use alcohol), normative beliefs, substance use-related expectancies, and intentions to prevent harm.

Results: Our analyses proceeded in three steps. First, we tested whether each lesson achieved an effect size > .2 for behavioral intentions at the 30-day follow-up. None of the lessons met this criterion. We then tested whether each lesson achieved an effect size > .3 for each corresponding mediating variable at the 30-day follow-up. The effects of the social norms lesson met this criterion. We then tested whether the expectancies lesson or the harm prevention lesson achieved an effect size >.3 for each mediating variable at the immediate posttest. Neither lesson met this criterion.

Conclusions: We conclude that the social norms lesson impacted normative beliefs, but we need to revisit our conceptual model to determine why the normative beliefs are not as strongly connected to behavioral intentions as expected.  We also conclude that the remaining two lessons need to be revised. We end with a brief discussion of a revision strategy based on these results and discuss how they impacted our third component selection experiment.