Evaluating the Effectiveness of Partnerships for Success in Reducing Community-Level Prescription Drug Poisoning Rates

The complexity of SAMHSA’s Center for Substance Abuse Prevention’s Strategic Prevention Framework Partnerships for Success (SPF-PFS) initiative contributed to the use of multiple types of quasi-experimental designs and data sources to evaluate PFS’s effectiveness in reducing prescription drug misuse at the State and community levels. The PFS national cross-site evaluation used the National Poisoning Data System (NPDS) as one key data source for outcomes. NPDS includes more than 60 million poisoning exposure case records, covering all 50 states, the District of Columbia and six territories. To assess the effectiveness of the PFS intervention, we compared differences in changes in poisoning outcomes between communities that implemented PFS and communities that did not implement PFS. In so doing, we implemented a non-equivalent control groups (NECG) design, propensity score weighting in the R package ‘twang’ and generalized non-linear mixed effects models in SAS Proc NLMIXED.

To convert individual poisoning cases to community-level poisoning rates, analysts (1) aggregated poisoning cases up to ZIP-code-level counts; (2) summed ZIP-code-level counts to the county level and, (3) generated community-level poisoning rates per 10,000 youth for the following substances: stimulants, sedatives, opiates, anti-depressants, and ethanol along with a poisoning measure that combined all five categories. These rates covered each Federal Fiscal Year from 2012 through 2016, with different baselines defined for communities within the PFS 2013, PFS 2014 and PFS 2015 grantee cohorts.

Typically under NECG designs, because of nonrandomization, potential confounding variables may predict both group membership (PFS or non-PFS community) and outcome which, without some control of confounders, would lead to bias in intervention effect estimates and make distinguishing PFS’s effects from the effects of measured confounders impossible. Community-level confounders such as baseline poisoning rates, number of crash fatalities, DUIs, and drug and liquor offenses were used in the generation of community-level propensity weights using the R package ‘twang’. Prior to balancing, PFS/non-PFS baseline differences ranged from ds of |.124| to |.484|, while post-weighting balance indicators had the highest d among confounders at |.049|. In propensity-weighted outcomes analyses, differences in changes over time favoring PFS were observed for any poisonings (d = -.34), stimulants (d = -.39), anti-depressants (d = -.27), and ethanol (d = .24). In this paper, we demonstrate the utility of combining multiple archival sources of data and modern approaches to assess causal inference for community-level effectiveness analysis.