Replication Evaluation: From Science to Practice

Introduction

Few programmes approved by well-known clearinghouses have a second study confirming the findings of the primary research. Independent evaluations are even rarer. Where repeat evaluations exist, differences in study design, outcome measurement and sample characteristics lead to questions about whether they are sufficiently similar to allow findings to be combined.

In the practice world, public agencies are under increasing pressure both to deliver evidence-based programs (EBPs) and to evaluate the effectiveness of services they provide. But their priorities and expectations can be different to that of the scientific community. They want inexpensive research that answers a core question – does this program work in my context? Is there any common ground here?

Method

The paper will present the findings of an independent evaluation of three EBPs implemented in Birmingham, UK. Randomized controlled trials were conducted in real-world public service systems of the Incredible Years BASIC parenting program, the Triple-P Level 4 Group parenting program, and the Promoting Alternative Thinking Strategies (PATHS) social and emotional learning program.

Results

This presentation will share the results of these three replication trials and reflect on how the implementation, evaluation design and findings were similar to and different from prior (primary) research examining these programs.

Implications

The presentation makes a case for an approach to replication evaluation that meets both the standards expected by well-known clearinghouses of EBPs and the Society for Prevention Research for replication as well as the financial and contextual constraints of the systems and service agencies that seek to deliver EBPs efficiently.