The Extended Age-Cohort Design As a Useful Alternative for Evaluations of School-Based Intervention

Randomized Controlled Trials are often considered the “gold standard” for evaluation of the efficacy or effectiveness of  intervention programs, also when applied to complex social organizations.  As has been  pointed out in the research literature (Chatterji, 2007; Wolff, 2000), however, the key assumptions underlying RCTs, originally developed for a kind of “laboratory-like” situations, are often severely compromised when large dynamic organizations such as schools are the units assigned to the various treatment conditions   And the non-equivalent control group design with use of analysis of covariance, ANCOVA, to “control for “ initial differences among pre-existing groups  is usually an inappropriate and risky enterprise (Judd & Kenny, 1981;  Weinberg, 1979).  In this paper, the ‘extended age-cohort design’ (Olweus, 2005; Olweus & Limber, 2010) is offered as an alternative and the strengths and possible weaknesses of this design are discussed. The presentation will argue that this design can often be used to advantage instead of the two design types mentioned.  In the extended age-cohort design,  data from presumably equivalent (grade) cohorts of students from the same schools are compared at two (or more) time points.  One grade-cohort provides data for Time 1 (before intervention) and the other grade-cohort data for Time 2, typically one year later and after several months of intervention. Possible developmental or maturational changes are controlled by comparing age-equivalent grade cohorts from the same schools at the various time points. A considerable strength of the design is that the majority of the members in the various grade cohorts have been recruited from the same, usually relatively stable populations and have typically been students in the same schools for several years. The schools thus serve as their own controls which factor also increases power. A particular threat to consider in this design concerns the possibility that changes in an outcome variable at Time 2 are a consequence of some irrelevant factor concomitant to the intervention, implying a “history interpretation” of the results.  With  roughly similar cohorts of schools starting with the program in consecutive years, however, it is usually possible to decide if such effects are likely and to estimate the magnitude of such possible effects. Several large-scale evaluation projects of the effectiveness of the Olweus Bullying Prevention Program (OBPP) as implemented in several different countries including the US and Norway  will be used to exemplify use of the design, thereby also illustrating aspects of one of the key themes of the conference.