An Empirical Study of Implementation Quality of Prevention Programs in Croatia

A series of meta-analyses and detailed assessments of the strength of research findings for certain practices and programs may help a consumer, agency, or community to select a program. However, more data on program outcomes will not help implement that program: implementation is an entirely different process. Likewise, implementation of an intervention must be well defined and carefully evaluated with regard to its effects on its intended consumers (practitioners, managers, organizations or systems).

This paper will present results of the doctoral research which focused on implementation quality of the cohort of 24 mental health and preventive interventions. Study was conducted within the collaboration of University of Zagreb, Faculty of Education and Rehabilitation Sciences and the Region of Istria, Department of Health and Social Services and is the first study of implementation in Croatia. Presented study was a part of the wider project “Preffi: Quality assurance in the Region of Istria”. Main aim of this doctoral study was to monitor overall level and variability of implementation quality in the mentioned cohort of 24 programs which represent community interventions which are financed though the local tenders and public funds of the Region of Istria. Also, this study explored the relationship among implementation outcomes and program outcomes, i.e. to answer the question how program outcomes are affected by implementation quality.

Three different measures of implementation quality were constructed, including items covering factors which affect the implementation quality i.e. implementation drivers (attitudes towards the intervention, training and knowledge, support for implementer, monitoring system, implementer’s skill, program standardization) and items covering different aspects of implementation quality (fidelity, dosage, participants’ responsiveness, quality, perception of program impact). These are measures for monitoring the quality of implementation from the position of organization manager, from the position of program implementers and from the position of program participants. Implementation factors and indicators of implementation quality for all types of informants were assessed at two points: after the first third of program delivery and at the end of program delivery.

Paper will offer contributions and suggestions for implementation research field in general, especially regarding the community-based interventions which still have to be researched. Conceptual model of implementation will be presented, together with implementation quality measures which are proven to be valid and reliable. Hierarchical linear modelling was applied regarding the complexity of data. Paper will present significant predictors of implementation quality as well as relationship of implementation quality and program effects.