Outcomes and Preliminary Explanations From the SPF SIG Cross-Site Evaluation, Cohorts I and II

Abstract Body: The primary outcome questions of the cross-evaluation of SPF SIG Cohorts I and II are: 1) did SPF funding lead to community-level improvement on targeted population outcomes?, and 2) what accounted for variation in outcomes performance across funded communities?

 States and their communities targeted many outcome priorities, using a wide variety of measures. To date, the cross-site analysis has focused on 8 priorities selected by multiple (3 or more) communities in at least 2 states. Five of the 8 targeted middle and/or high school students (alcohol use, binge drinking, driving after drinking, riding in a car with a drinking driver, and marijuana use), 2 targeted  young adults (binge drinking and driving after drinking) and 1 targeted all ages (alcohol-related motor vehicle crashes, injuries, and/or fatalities). Where states provided multiple measures of the same outcome, measures were selected to maximize the commonality across states. 

 Preliminary analyses on 23 of the 26 states indicate that the majority of targeting communities experienced favorable changes (i.e., declines from pre-intervention to post-intervention) in most outcome measures assessed.  Two exceptions to this pattern were binge drinking among young adults (for which only 13 of 30 communities experienced a decline) and driving after drinking by young adults (9 of 23 communities).  Otherwise, declining (i.e., improving) prevalence rates was a robust finding across outcome measures and across targeting states and communities.  For example 91 of 125 communities (73%) that targeted underage drinking experienced a decline in 30-day alcohol use by students.  Of those, 53 were statistically significant, compared to 12 communities with a statistically significant increase.  These results must be viewed in light of secular trends (e.g., the overall national trend of falling underage drinking rates during the observation period), and by themselves do not provide compelling evidence of SPF SIG impact.  However, among states that provided data on non-funded comparison communities (some matched and/or randomly assigned, others not), comparative tests were more than twice as likely to significantly favor targeting communities as non-targeting or non-funded communities.  

 While statistical tests were not applicable for alcohol-related motor vehicle crash data, which typically cover the entire target population, 5 of 5 states targeting this priority showed nominal pre-post improvement among targeting communities, and the 2 states with comparison communities both showed greater comparative improvement.

 The SPR presentation will explore these data more comprehensively through multilevel longitudinal modeling, which will also incorporate intervening variables and other state and community characteristics (e.g., demographic characteristics, levels of and changes in prevention infrastructure, implementation fidelity at both state- and community levels), in efforts to identify key factors that predict success in reducing community-level prevalence rates and other targeted outcomes. Limitations and preliminary lessons learned will be discussed.