Transportability in Encouragement-Design Interventions

Introduction: Multi-site interventions are common in public health. In analysis of these interventions, site is typically controlled as a fixed effect. This approach assumes that the effect of the intervention does not differ by site. When variation in the intervention effect is considered, usually interaction terms between intervention and site are added. This approach assumes that the intervention effect is site-specific, and not transportable. However, an intervention effect that differs by site could be explained by differing distributions of population characteristics across sites. If variation in effects by population characteristics within site were captured, effects might be transportable. The goal of this work is to better understand contributors to variability in effect estimates across trial sites. 

Methods: We motivate this problem with the Moving to Opportunity (MTO) study, an encouragement-design intervention in which families in public housing were randomized to receive housing vouchers and logistical support to move to low-poverty neighborhoods. This intervention took place in 5 cities across the U.S. and intervention effects varied by site. To date, there has been no quantitative examination of the underlying reasons for these site differences. 

Our objective is to examine underlying reasons for site differences in the MTO intervention effects on societal attachment and mental health outcomes among adolescents. Specifically, we wish to test the null hypothesis that the predicted effect estimate for City B = the true effect estimate for City B, where the predicted effect estimate borrows the conditional outcome model from City A and makes use of differing distributions of population characteristics between City A and City B through Pearl’s transport formulas. If we fail to reject the null, this suggests that the intervention may be transportable based on the covariates included in the transport formula. If we reject the null, it suggests that the intervention is not transportable given our measured covariates. 

Results: We develop and employ a targeted minimum loss-based estimator (TMLE) for two predicted estimands: the intent-to-treat average treatment effect and complier average treatment effect. We illustrate how a TMLE can incorporate a set of pre-treatment variables and Pearl’s transport formulas to estimate expected effects across sites in the MTO data. 

Conclusions: When interventions are implemented across sites, expectations of site-specific intervention effects should reflect the distribution of relevant population characteristics at that site.