Optimizing the Use of Free Online Technological Tools to Monitor Prevention Program Implementation and Outcomes in Chile

Introduction:

Implementing evidence based interventions with fidelity to the original model has been a challenge in Latin America. As more foreign interventions are disseminated in the region, it is essential to ensure that the intermediate and final outcomes are achieved, to ensure the intended impact. The development of local preventive initiatives also requires monitoring and evaluation to determine if goals are being met. Establishing a monitoring and evaluation process to obtain real-time information on program performance has become increasingly necessary. In response, we have developed a performance management tool that uses free online software for data collection and visualization, which allows organizations and programs to monitor their interventions at low cost.

Method:

We developed an implementation monitoring process that can be used for any intervention. Evaluation professionals who are familiar with the tool meet with local program staff to jointly : (1) develop a theory of change that identifies the causal relationships between activities, immediate products, results, impact and the expected long-term goals; (2) prioritize key processes in the implementation and outcome goals; and (3) develop performance indicators and data collection tools. This information feeds into a dashboard that uses free online software (Data Studio and Google Forms) and allows program staff to visualize the results and the causal relationships identified in the theory of change. The dashboard allows program implementers to determine the need for corrective measures for implementation using real time data.

Results:

We present two examples of interventions that have used this program monitoring system. “Familias Unidas,” an evidence based intervention that prevents risky sexual behavior and substance use, utilized the system to monitor fidelity to the original model in their Chilean implementation. “Revinculación Familiar”, a Chilean intervention designed to help families in the child welfare system stay connected, has used this technological tool to observe and analyze de causal mechanism behind the intervention model. In both cases, programs took corrective measures based on the data collected, such as modifying supervisory strategies, or providing additional training to program facilitators.

Conclusions:

Technology can play an important role in providing low cost tools that facilitate the analysis and communication of implementation and outcome data for preventive programs, without adding a big cost and time burden to purveyors. Monitoring implementation fidelity of existing evidence-based interventions, and monitoring outcomes of interventions for which effectiveness-data is not yet available, can help us optimize resources and achieve better intervention results for preventive programs in the region.