Using Multilevel Regression Mixture Models to Identify Heterogeneity in Contextual Effects

Prevention scientists are often interested in contextual risk and protective factors which impact groups of individuals, such as the effects of neighborhood and school norms. Theory often suggests that not all individuals in a group are impacted in the same. However, existing regression mixture models have not yet been adapted to examine this type of ‘cross-level’ heterogeneity. In this study we propose a new multilevel regression mixture model which is capable of detecting cross-level differential effects. This approach allows for individuals within the same cluster (community or school) to be affected by the community and school in different ways as evidenced by membership in different latent classes.

In this presentation, we propose a novel formulation of the multilevel regression mixture models that allows heterogeneity in contextual effects. Specifically, we examine the efficacy of multilevel regression mixtures to identify systematic differences in the effect of a level-2 (or cluster-level) variable on the outcomes of individuals withina given cluster. The first aim of this presentation is to use Monte Carlo simulations to demonstrate the feasibility of using multilevel regression mixtures to identify differential effects of a cluster-level variable on individuals within a cluster as a function of sample size. We then demonstrate this method by examining heterogeneity in the effects of teaching style on individual students using data from over 800 classrooms across the US.

Results show that when heterogeneity in cluster level effects exist, regression mixtures can find the true number of differential effects (using the BIC and adjusted BIC) in 90% of the replications with a sample size of 2500 students and in every replication with 5000 students. Given that the true differential effects are found, we examine bias in parameter estimates and coverage of the estimated standard errors. No substantial bias was found in any model parameter, coverage estimates for the 95% confidence interval were somewhat lower than 95% for small samples but were over 89% and thus still useful. Overall, simulations show that the multilevel regression mixture model is able to find heterogeneity in cluster level effects with sample sizes seen in large multilevel studies. Real data example is also presented to demonstrate the utility and the procedures for estimating this approach. Implications and analytic suggestions will be discussed.