Population-Based Outcome Measurement Through the Strategic Prevention Framework (SPF): Reflections On Colorado's Experience

Co Author: Suzanne Kennedy-Leahy

ABSTRACT BODY:

SAMHSA’s Strategic Prevention Framework (SPF) initiatives introduced an important shift in national prevention policy. Previously, the field had largely focused on program outcomes measured at the individual level. SPF initiatives launched an emphasis on population-based outcomes measured at the community level. This national policy created a framework and allocated significant resources that enabled researchers to explore an area of measurement that was understudied and rarely assessed. However, roll-out of initiatives seeking to impact community level change that require new measurement, like community level outcomes, take time as states, communities, and researchers work to change practice and develop infrastructure. In Colorado, a SPF SIG Cohort I state, it took three years for the state and communities to implement Steps 1-3 of the SPF model. It was not until year four of the grant that community strategy implementation and outcome evaluation began. This timeline allowed only two years for outcome measurement and observation of change. Further, gathering baseline and outcome data requires time, local partnerships, and state support. Colorado is a “local control” state that gives authority to school boards to determine policies concerning survey participation. Although a number of communities had experience with the modified outcome measurement tool (youth risk behavior survey) prior to implementing SPF, participation had not been consistent over time nor large enough to be generalizeable to the “community level.” Baseline had to be established over the course of several years and data collection required partnership building with schools, which proved challenging given varying school policies and school official turnover.

 Evaluators documented positive changes in Colorado communities, like passage of social host liability ordinances and development of high-functioning community coalitions, which the literature suggests lead to successful outcomes, yet community impact was not observed as change in the outcome measurement tool. Analysis of the youth survey outcome data demonstrated no practically significant outcome changes. Federal cross-site evaluators offered to support Colorado in gathering data several years after the grant ended to determine whether communities were still implementing prevention strategies and to assess effects after a longer period of time had elapsed. However, Colorado was unable to gather data because there is no policy support statewide for conducting surveys. These outcome results raised interesting questions for Colorado regarding the timeline needed to measure community level impact and the time needed for local policy-makers to implement changes recommended by national policy frameworks, among other questions.