Measurement and Psychometric Issues in Statistical Mediation Analysis

Introduction: Statistical mediation analysis is a common approach used in prevention research to evaluate an intervention designed to target mediators that are thought to be causally related to an outcome. One of the critical assumptions of statistical mediation is that the model contains reliable and valid measures of the mediator. However, despite prior research on statistical mediation, the consequences of not measuring the mediator accurately have not been extensively investigated. The purpose of the poster presentation is to discuss three measurement problems in the statistical mediation model, where the first two problems will be theoretically conceptualized and one will be expanded fully through a Monte Carlo simulation. Problem one discusses issues with unreliable measures, which could be addressed through latent variable modeling to account for measurement error. Problem two discusses the issue where treatment and control conditions might have different latent factor structures in the mediator, which can be assessed with tests for measurement invariance. Problem three discusses an issue where only a facet of a construct (a specific part of broad construct) is the true mediator and not the construct as a whole. Identifying the important facets of the mediator would help researchers save resources by not implementing complete intervention programs when only part of them previously worked.  The bifactor psychometric measurement model can be considered as an approach to simultaneously estimate the broad and facet variance of a construct, although its estimation properties in the mediator model have not been investigated.  

Method:Monte Carlo simulation methods were used to investigate the statistical properties of the single mediator model, where the mediator was measured with nine items, modeled with a bifactor measurement structure and the mediated effect is through a facet of the mediator.  The Monte Carlo study included the factors of sample size, effect size in the mediated effect, and different proportions of broad construct and specific facet variance in the mediator.

Results:Preliminary results suggest that the statistical mediation model with a bifactor structure in the mediator has Type 1 errors below the nominal value across all conditions. Also, the model achieves appropriate power (above .80) and coverage (around 95%) through the distribution of the product method and Monte Carlo confidence intervals with as little as 200 cases, small facet variance, and medium effect sizes for each of the paths.

Conclusions: Results suggest that a statistical mediation model with bifactor structure for the mediator can be used to accurately detect effects in conditions found in prevention literature.  This abstract raises awareness of the importance of measurement theory in the mediation model.