A Comparison of Bootstrap Confidence Intervals for Multilevel Mediation

Introduction:Clustered data are present in a large proportion of prevention intervention studies with participants nested within a variety of geographical or organizational groups, including classrooms and hospitals. At the same time, mediation models are widely used to examine the underlying mechanisms of change in these same prevention interventions. Much of the recent work on mediation models has recommended the use of resampling methods (e.g., bootstrapping) to create confidence intervals around mediated effects due to the nonnormality of the mediated effect in most situations. While existing work has examined the use of resampling methods in multilevel data, most of this work has focused on the residuals from individual models. The most common multilevel mediation model for prevention is the 2-1-1 model, however, which requires the estimation of two separate multilevel models. While these equations may be estimated simultaneously using the multilevel structural equation modeling framework, there are situations where remaining in the multilevel framework is preferable, such as more than one level of clustering. The current study examines how the residuals from multiple multilevel models may be used to create confidence intervals for multilevel mediation effects.

Methods:A simulation was conducted to compare seven different approaches to creating confidence intervals for 2-1-1 multilevel mediation models: 1.) resample cases ignoring clusters, 2.) resample only clusters, 3.) resample cases within each cluster, 4.) resample clusters first, then resample cases within the selected clusters, 5.) use the distribution of the individual parameters (parametric bootstrap), 6.) use the distribution of the Level 1 and Level 2 residuals (parametric residual bootstrap), and 7.) resample the Level 1 and Level 2 residuals (nonparametric residual bootstrap). Three different intraclass correlation effect sizes were investigated. The methods were compared in terms of statistical power, Type I error rate, coverage, interval width, and bias in estimating the intraclass correlation.

Results:The simulation showed several differences in performance among the seven methods, particularly differences in the residual bootstrap methods compared to the nonresidual methods. These differences were then illustrated using empirical data from a large drug prevention and health promotion intervention for student athletes.

Conclusions: Multilevel data can no longer be used to justify the use of normal theory based approaches for testing mediation when resampling approaches exist.