Shared Measurement Strategies for Leveraging Big Data to Showcase Outcomes

In 2015, South Florida Behavioral Health Network (SFBHN) released funding for an evaluation of the Substance Abuse Block Grant funded youth prevention programs. BSRI successfully applied and was funded to evaluate the Prevention System of Care (PSOC). The PSOC seeks to make an impact on youth and families in Miami-Dade and Monroe Counties that includes de­creasing the number of youth who use drugs; decreasing risk factors that are associated with use; increasing protective factors known to mitigate use; and enhancing community resources to build capacity of local communities to en­act sustainable progress. This presentation includes both the funder (SFBHN) and evaluator (BSRI) perspective to discuss the use of systems-level data to showcase prevention outcomes as a way to implement data-based decision making in funding prevention efforts.

Although longitudinal program-level data is captured by prevention providers for individual youth participants, there is no comparison group to assess intervention/programmatic changes against. Additionally, individual program pre-post data is not comparable across providers implementing various programs, or at the County-level/State-level as risk and protective factors are assessed in different ways. With funder support, BSRI was able to implement shared measurement across 9 different evidence-based programs through the development of a standardized evaluation tool based on the Communities that Care (CTC) youth survey and risk and protective factor framework (Hawkins & Catalano, 1992). The PSOC evaluation was also able to include a proxy-comparison group, as the State implements a Youth Substance Abuse Survey taken from the CTC survey. From this State-level implementation, the most localized data available is at the County level. Therefore, using a standardized tool based on the same survey allowed BSRI to detect zip-code-level differences in risk and protective factors, identify which evidence-based programs were most effective at targeting specific risk and protective factors, and determine how those youth targeted by the PSOC compared with the general County-wide CTC survey sample regarding risk. In addition to state and county-level comparisons, a modified version of the CTC survey is used by multiple states and in communities that adopted the CTC evidence-based community-level program.

This presentation offers a unique perspective of using standardized data/shared measurement as a strategy for leveraging broader data systems to drive evaluation. Both the funder and evaluator experiences through this process will be shared with the hope that participants will have a clearer idea about how to enact this strategy in their own evaluation or service system work.