Cross-Validation of the Teacher Observation of Classroom Adaptation-Revised (TOCA-R) Instrument with Classroom Observation Data in the Context of a Good Behavior Game Trial

Introduction: Aggressive and disruptive behaviors, as early as first grade, are confirmed antecedents for social and behavioral problems later in life. The Good Behavior Game, a classroom-based behavior management strategy is one of the few preventive interventions that have been found to have positive short, mid, and long-term impacts including reductions in alcohol and drug abuse/dependence. When determining program effectiveness, it is essential to measure child behaviors accurately. In this paper we describe two measures used to capture child behaviors and examine how the measures correlate at the child and classroom levels: 1) the Teacher Observation of Classroom Adaptation-Revised (TOCA-R), which measures teachers’ ratings of children adaptation to task demands of being a student and 2) an observation protocol that captures student and teacher behaviors as well as classroom level measures of classroom management.

Method: The TOCA-R interviews and classroom observations were conducted in 54 first-grade classrooms during school year 2010-11.  The total student sample includes 1,027 children (girls: 48.81%; Hispanics: 75.00%) . Teachers were interviewed twice a year (fall and spring) and classrooms were observed six times a year (twice in fall, winter and spring).  The teacher interviews lasted about two hours. Teachers rated each child on 57 behavioral indicators; constructs include aggressive, disruptive behavior, prosocial behavior, emotional regulation, and academic readiness.  Each classroom was observed by two observers, each observed half the students (maximum 15). Each minute, instances of off-task, disruptive, verbal or physical aggression, or socially isolated behavior were recorded. The child exhibiting the behavior was noted. 

Results: We used the direct observations to cross-validate the TOCA-R instrument.  According to preliminary analyses students who exhibit aggressive or disruptive behavior during observations are more likely to be rated as disruptive or aggressive by their teachers.  In addition, girls have better behavioral outcomes than boys, and children classified as Hispanic also tend to have better behavioral outcomes, whether measured by TOCA-R or direct observations.

Discussion: Implications for research and practice will be discussed including the tension in the field between teacher reports and observations collected by independent raters. The TOCA-R, which can also be reliably administered as a survey is more cost efficient and less burdensome to administer than observations, relies on teacher self-report. The TOCA-R has been used as an outcome measure in research trials for decades. In its survey form it could be used as part of a broader measurement system to identify students’ needs and responses to interventions.