Implementation Evaluation Findings for Safe Public Spaces in Schools

Introduction. The understanding of implementation is critical for any randomized trial so that variation in impact may be fully understood. The implementation evaluation for the Safe Public Spaces (SPS) trial addressed four research questions: (1) How is SPS integrated into and coordinated with pre-existing programs and aligned to school safety and discipline policies and practices? (2) To what extent is implementation consistent with the design of SPS as specified by the developer? (3) How do SPS schools resolve challenges to program implementation? And (4) To what extent do SPS and control schools differ on practices/principles aligned with the SPS program?

Methods. In the 2016–17 school year, schools were visited twice each year (once each in the fall and spring). Visits included interviews with principals/administrators (30 minutes each) as well as 2–3 hour structured observations. In the 2017–18 school year, to increase representativeness of the data, the study team visited all participating treatment and control schools four times, twice in the fall and twice in the spring, with the first and last visits including observations and interviews, and the other two visits including structured observations only.

Results. Year 2 results will be presented at the conference. As of this date, results at the end of Year 1 of intervention showed that staff in SPS schools were more than twice as likely to collect safety data beyond what was required by the district, and were twice a likely to review those data with staff. In SPS schools, there were fewer behavior incidents observed in the cafeteria or during transition between classes, but the rates were similar for hall walking. Staff in SPS schools were more likely to give effective reminders and directors than were staff in control schools, and they were less likely to give ineffective reminders or directives.
Staff in SPS schools were most likely to report changes to lunch transition, changes to hallway procedures (e.g. “walk to the right”), incentives for teachers to participate in public space safety, and changes in cafeteria procedures (separate lunch times for boys and girls). There were no notable differences between SPS and control schools in addressing emotionally charged incidents.

Conclusion. All schools address school safety, and at the more severe end of the student behavior continuum, there were few differences between how SPS and control schools managed safety. For routine, everyday events, there was greater differentiation between study conditions. There was variability in implementation quality, with a minority of schools implementing the intervention at a high level of fidelity. Year 2 findings will be used to contextualize overall study results.