Latent Transition Analysis Versus Traditional Methods for Assessing Clinical Significance

Introduction:  Statistical significance and effect sizes fall short in evaluating interventions’ practical impact. Underutilized in the prevention literature, methods for determining clinical significance (CS) fill this gap by identifying participants as treatment successes or failures. This study proposes use of Latent Transition Analysis (LTA) as an approach for evaluating CS and contrasts it to a traditional method developed in psychotherapy research. 

Methods: In 5 states, 2,717 drivers arrested for operating under the influence participated in PRIME For Life® (PFL), a group-delivered, indicated prevention program. Preliminary analysis showed statistically significant baseline to posttest improvements on outcomes.

Results: Based on two risk perception and two behavioral intention outcomes, LTA identified four classes at baseline and at posttest. Those in the low risk (LR) class  at baseline (less than a fifth of the sample) had >80% probability of remaining in that class at posttest. Over half of the sample was in one of two moderate risk classes at baseline, and they most commonly transitioned to the LR class. While only a fifth of those in the high risk (HR) class at baseline remained HR at posttest, they were more likely to transition to the moderate risk classes rather than the LR class. In separate analyses, we then estimated CS for each of the four outcomes using Jacobson et al.’s procedures. These define improvement and deterioration as changes that are reliable (not due to measurement error) and that cross cutpoints differentiating healthy versus unhealthy functioning. Findings varied across the four outcomes. At baseline 45%-67% were above the cutpoint (HR), and 33%-55% were below the cutpoint (LR). Depending on outcome, among HR individuals 17-58% improved such that they crossed the cutpoint to LR, 4-10% improved but did not cross the cutpoint, 30-77% did not change, and ≤3% deteriorated.  Among LR individuals, 5% improved, 92-97% did not change, and ≤3% deteriorated.  

Conclusions:  In prevention research, the CS concept provides unique and useful information about an intervention’s practical utility. LTA provides an advanced approach to evaluating CS, but researchers must choose which approach is best suited to their particular application. The methods differ in that one is univariate and variable-centered, and the other multivariate and person-centered. They provide alternative types of information, handle measurement error differently, and vary in whether predictors of transitions are accounted for.