Scaling-up Evidence Based Practices in Education through State-Wide Collaborations

Introduction: An integrated tiered system of behavior and literacy supports was implemented in 25 school districts across the state of Oregon through a professional development initiative. The Effective Behavioral and Instructional Support Systems (EBISS) initiative trained district level leadership teams to integrate School-wide Positive Behavior Intervention and Supports (SWPBIS) and the School-wide Reading Model (SWRM) from the district office to the individual classroom. The EBISS initiative was built upon strong collaborations between the department of education, higher education, education service districts, and schools districts. These partnerships sustained and extended beyond the duration of the initiative.

Methods: To examine implementation impacts and changes from Year 1 to Year 4, we relied primarily on two types of analyses.  First, for dichotomous data, such as whether schools had a behavior leadership team in place, we computed a standard χ² test of dependence.  This determined whether there was a dependency between the proportion of schools with a behavior leadership team and study year.  The test involves a two (in place vs. not in place) by four (years) table with the number of schools provided in each cell. Second, for tests of continuous measures, such as SET scores or the mean percent of students who met the state standard on the academic outcomes, we examined gains from Year 1 to Year 4 with tests of growth over time. We fit the data to growth models with SAS PROC MIXED version 9.2 using restricted maximum likelihood (REML), generally recommended for multilevel models.

Results: Proximal outcomes included gains in building level SWPBIS and SWRM implementation teams. Distal outcomes included statistically significant gains in oral reading fluency (ORF) in first and third grades and marginally significant decreases in the percentage of students in the intensive category in second and fourth grades. Changes observed on state assessments in grades 3 – 5 were not statistically significant.

Conclusions: To help characterize the overall potential for impact, we use the RE-AIM framework, which offers a five-dimension evaluation schema for implementation or dissemination efforts: reach, effectiveness, adoption, implementation, and maintenance. The RE-AIM framework, originally developed to assess the public-health impact of interventions, is particularly useful for guiding and evaluating efforts to implement and disseminate empirically supported interventions, and has recently been discussed in education and school psychology. These findings and implementation considerations provide a model building large-scale partnership and for scaling up systems of evidence-based practices.