Assessing Implementation Leadership in Educational Settings

Introduction: Leadership has been identified as an important contextual factor related to implementation success (Durlak & Dupre, 2008). Until recently, there were no empirically validated, brief measures that assessed organizational leadership for implementation (Aarons, Ehrhart, & Farahnak, 2014). The Implementation Leadership Scale (ILS; Aarons et al., 2014) was developed to understand and improve strategic leadership behaviors for implementation success. Thus far, the ILS has been used primarily in health and allied health care settings (Aarons, Ehrhart, Torres, Finn & Beidas, 2016; Aarons et al., 2014). Implementation leadership, however, is also a critical component of implementation in schools (Forman et al., 2009). We are unaware of any studies using the ILS in schools. The current study uses the ILS to assess teacher perceptions of implementation leadership. We describe levels, reliabilities, between-site variance, and changes over the school year in implementation leadership.

Methods: Data for this study are from the TOOLBOX Implementation Research Project (TIRP). TOOLBOX (Collin, 2015), a school-based prevention program, was implemented in four elementary schools by 95 teachers (95% female, 62% European, 11% Hispanic, 7% African American). The ILS was completed (i.e., no items missing) by 63 teachers in October and 45 teachers in both October and May. There were no significant differences in race, gender, or initial ILS scores between teachers who completed a second ILS and those who did not.

The ILS yields a Total Scale and four sub-scales scores: Proactive, Knowledgeable, Supportive, and Perseverant Leadership. The ILS data collection instrument and scoring mirrored the process of the developer (Aarons et al., 2014), rating items on a 0 (“Not at all”) to 4 (“To a very great extent”) scale. Paired sample t tests and hierarchical models were used.

Results: In October, teachers perceived implementation leadership to be “moderate” to “great” (e.g., Proactive Leadership M: 2.15, SD: .85; Supportive Leadership M:2.72 , SD: .78. In May, teachers reported significantly lower levels of implementation leadership (p <.02; d = .34-.72). Alphas ranged from .78-.89. ICCs ranged from .10-.23. Teacher ratings in October were significantly associated with ratings in May across all scales (p < .001).

Conclusions: Levels of implementation leadership found in this school-based study were similar to those reported in mental health clinics (Aarons et al., 2014). The scales showed acceptable internal reliability. Ratings were significantly related over time, although teachers generally perceived small to medium declines in implementation leadership over the school year.