Dissemination of a Statewide, Juvenile Justice Program Improvement Initiative Based on Meta-Analytic Findings: Partnerships, Potholes, and Preliminary Data on the Path of Going to Scale

Pennsylvania is one of four states engaged in piloting the Standardized Program Evaluation Protocol (SPEP; Lipsey, 2010). SPEP is a scoring scheme, based on meta-analytic findings, for juvenile justice programs. It represents an alternative angle on what can be considered “evidence-based” and provides a metric for establishing confidence in program effectiveness. Pennsylvania has adopted SPEP as a complementary approach to its statewide dissemination of evidence-based programs.  

The SPEP scheme evaluates programs on four aspects: service type, quality of service delivery, dosage and duration, and risk level of youth served. Each of these components draws on standards common to evidence-based programs, including formalized training, manualized delivery protocol, mechanisms for observing and responding to drift, achieving target amounts of service, and providing the program to appropriate youth populations. Data for scoring are derived from interviews with service providers, attendance rosters, and the juvenile court data system. After scoring, program providers and probation officers develop and implement an improvement plan to strengthen the program. Technical assistance is provided by a state-level intermediary organization with expertise in supporting evidence-based programs.

The statewide roll out of SPEP requires buy-in from and partnership with each of the state’s 67 counties, partnership across state-level systems, and coordination by a state-level advisory group. A decentralized juvenile justice system combined with the qualitative aspects of SPEP has implications for taking it to-scale within the state. The primary key to going to scale surrounds the need to develop and sustain a robust “learning community”, which shares knowledge, direction, and insight into SPEP and the roll-out process.

The roll-out initiative has been under way for 10 months and preliminary SPEP score data have been gathered from four pilot counties on community-based and residential juvenile justice services, including state-run youth development centers and forestry camps. Trends in program scores and targeted improvement areas will be shared, along with lessons learned in state and county partnerships and learning community development.