Cross-Synthesizing Quantitative and Qualitative Evidence to Optimize Systematic Review Results for Decision-Makers

In educational settings, challenging behaviors are problematic because they are associated with stressed teacher-child relationships, increased chances of children dropping out of school, and predictive of teacher burnout. When supporting children’s behaviors in schools, there is a need to understand both child- and teacher-level effects and experiences of the process. Establishing effectiveness of prevention programs prior to wide-scale implementation is necessary but not sufficient, because decision-makers consider a range of issues before take-up. This paper illustrates how using mixed methods to systematically review a teacher classroom management program provides more relevant results for educational decision-makers working to prevent behavioral problems.

We systematically searched educational and psychological electronic databases and key websites to identify and retrieve studies examining the Incredible Years Teacher Classroom Management (IY TCM) program. Two independent reviewers screened studies for relevance. To minimize potential bias in the meta-analysis from traditional approaches of selecting only one effect per study, analyses used the metafor package in R Studio to model a multilevel meta-analysis and included all relevant effects to estimate an overall effect. Qualitative data were extracted into NVivo 11 for a grounded theory analysis, operating inductively and deductively to focus on processes experienced by stakeholders. Quantitative and qualitative findings were then cross-synthesized using framework analysis and integrative grids.

Nine studies were included from England, Ireland, Jamaica, United States, and Wales. Quantitative effects supported IY TCM as improving teacher management strategies and child behaviors. Teachers qualitatively reported experiencing benefits to their own increased knowledge, locus-of-control, emotional wellbeing, and practice. They also reported feeling that children benefited behaviorally, socially, and emotionally. Trial and experiential findings were generally harmonious, although qualitative findings suggested a broader conceptualization of benefits than were quantitatively measured and illustrated key processes in implementation.

Systematically reviewing programmatic evidence from effectiveness trials alongside data on teachers’ experiences produced results more relevant to and useable for decision-makers. By cross-synthesizing international findings on IY TCM, this paper illustrates how practitioners’ experiences can triangulate research findings and also contextualize effectiveness and implementation information for different systems.