Optimizing the Effectiveness of a Role Model Intervention Using a Factorial Experiment with a Mediator

Introduction: Mediation analysis contributes to program improvement by identifying the causal intervening mechanism. In this study, we combine mediation analysis with an optimization strategy to determine the most effective version of a brief social psychological intervention to reduce gender achievement gap in STEM. In the literature, stereotype relevant worries have been suggested as partly responsible for this gender gap. Specifically, when someone is concerned that her actions will represent her gender group, this creates a threat on identity and in turn undermines her task performance. Accordingly, past work has shown that in-group role models buffer stereotyped targets from stereotype threat. In this study, we test whether a role model intervention reduces gender gap in STEM through decreasing perceived stereotype threat. Using a factorial experiment, we manipulate the three critical features of the role model intervention: role model’s orientation (both success and failure oriented versus only success oriented), the perceived attainability of the success of the role model (star-target versus non-star-target) and the similarity of the role model to the participant (more similar versus less similar).

Methods: The study uses a fully factorial 2x2x2 between-subjects experimental design in which 210 female engineering college students are randomly assigned to read one of the eight possible versions of a letter from a female role model in their respective field. We then assess their status on the mediator, which is the perceived stereotype threat. Participants then take a math performance test and a mental rotation task. Structural equation modeling is used to test mediated effects. Bias-corrected confidence intervals are estimated using 1000 bootstrapped samples.

Results: We test our hypothesis that female engineering students’ performance will be higher in the condition where the role model is both success and failure oriented, a non-star-target (i.e., presenting attainable success), and similar to the participants in terms of age. In presenting the results, we focus on how the method used allows for the refinement of the intervention by shedding light on its underlying mechanism of change.

Conclusions: As there is little guidance in the literature on using this combined method, we use this case study to both illustrate its use and discuss its quantitative challenges.