Inverse-Propensity Weighting Approaches for Estimating the Mediated Effect in Pretest-Posttest Control Group Design

Introduction:Pretest-posttest control group designs are common in prevention research where there is random assignment of units and a mediator and outcome are measured before and after the intervention is delivered to one group.  The purpose of this abstract is to describe modern causal inference methods applied to this common prevention design.  Estimating mediated effects in prevention research helps determine what aspects of prevention programs were effective but are still prone to confounder bias of the mediator-outcome relation.  Many traditional methods exist to adjust posttest scores based on pretest scores but little work has investigated modern causal inference methods to handle pretest scores and confounder bias of the mediator-outcome relation.  One such modern causal inference method is Inverse-Propensity Weighting (IPW).  IPW can handle the pretest score on the mediator and outcome by inversely weighting each observation by the probability they have a specified score on the mediator at posttest given pretest values of the mediator and outcome.  Because mediators are often continuous in prevention research, IPW weights can be unstable therefore it is important to investigate various methods of creating IPW weights such that they are stable. We seek to investigate the conceptual basis of these methods and assess the performance of the methods in a simulation study.

Method:A Monte Carlo simulation study was used to investigate the statistical performance and confidence interval coverage of the different IPW methods for pretest-posttest designs.  The Monte Carlo study consisted of a sample size often found in prevention research: 200 and parameter sizes of the mediated effect corresponding to zero, small, medium, and large effect sizes.  

Results: Results suggest different percentile truncation of the IPW weights do not have an effect on Type 1 error rate (all below alpha 0.05) or power to detect a significant mediated effect.  The different methods of percentile truncation of the IPW weights did have an effect on the 95% confidence interval coverage of the mediated effect. IPW with 90^thpercentile truncated weights had confidence interval coverage closest to 95% than any of the other IPW methods but this coverage became smaller as effect size of the mediated effect increased.

Conclusions: Results suggest that IPW with 90^th percentile truncated weights performs the best in assessing mediated effects in pretest-posttest designs.