Monitoring the Theory of Change in the Pilot Implementation of the “Communities That Care” Prevention System in Chile

Introduction: Three communities in Chile have been implementing an adaptation of the evidence-based Communities that Care (CTC) prevention system aimed at preventing youth problem behaviors, such as substance use. CTC provides training and technical assistance, and empowers local coalitions to assess and prioritize the community’s prevention needs, select evidence-based prevention programs matched to priorities, and deliver selected programs with fidelity and reach. As part of the process evaluation of CTC’s pilot implementation in Chile, we used a set of monitoring tools to assess whether key elements of CTC’s logic model were being implemented as expected.

Methods: We monitored CTC implementation using the Milestones and Benchmarks Tool, a checklist of key implementation tasks to meet within each CTC phase. We monitored coalition development with the Community Board Interview (CBI), a survey of CTC coalition members that examines board functioning and capacities, in 2016 (N=61) and 2018 (N=59). We monitored systems change using the Community Key Informant Interview (CKI), measuring the prevention environment (e.g., key leader support for prevention, community norms towards youth drug use, use of a science-based approach to prevention) in 2015 (N= 46) and 2018 (N=63). Finally, we monitored CTC outcomes (youth risk and protective factors and problem behaviors) by administering the CTC Youth Survey (CTCYS) to students in grades 6-12, in 2014 (N=2145), 2016 (N=2227) and 2018 (N=1846).

Results: Sites met almost 90% of the milestones in phases 1-4, and are currently in phase 5. CBI results showed coalition involvement and cohesion, with members reporting benefits to their CTC participation and overcoming important barriers to implementation. CKI results showed that community norms for youth alcohol consumption were stricter than for tobacco smoking and appeared to be getting less permissive over time. CTCYS results showed that one of the most elevated risk factors was community disorganization, one of the most depressed protective factors was youth prosocial involvement and some of the most salient problem behaviors were alcohol, cigarette and marijuana use.

Conclusions: Evaluating process and monitoring a preventive intervention’s theory of change is key to tracking progress and ensuring fidelity. CTC prevention coalitions in Chile were able to meet implementation tasks, develop functioning coalitions, engage key leaders in the community, and sustain the assessment of youth risk, protection and outcomes in their communities. Since the implementation of programs to address priorities has only recently begun, we did not yet expect to see changes in youth outcomes (e.g., reductions in drug use or risk). However, the installation of the youth survey is a crucial step for sustaining CTC long term, as this tool provides the data needed to select priorities and interventions, and allows monitoring of outcomes and change over time.