Methods Toolbox for Meta-Analysis of Individual Participant Data: Examples from Project Integrate

Introduction: The use of individual participant data from multiple intervention studies especially in the context of evidence synthesis represents a major methodological innovation. However, despite increasing interest in standardizing and pooling individual participant data from multiple sources to achieve the scale and diversity for mechanism-focused investigations and broader and more robust generalization, respectively, research applications have been limited. Project INTEGRATE is an ongoing comprehensive meta-analysis project in the field of brief alcohol interventions for adolescents and young adults. It was originally conceptualized within the Integrative data analysis (IDA) framework. The term IDA was coined to provide a rationale for pooling data from multiple independent studies and analyzing them as a single data set. Although the premise of IDA still remains valid, we have come to view IDA as one specific method within the broader context of meta-analysis. We present an overview of our analytic approaches and invite prevention researchers, meta-analysts, and methodologists to our “back room” for discussing challenges and methods.

Methods: Based on our experience of analyzing individual participant data for the past 10 years for Project INTEGRATE, we present three major analytic approaches to validly analyze individual participant data from multiple sources in data applications.

Results: The first approach is to combine data from different studies into a single data pool while treating study membership as a fixed effect and adjusting for standard errors using a sandwich estimator to adjust for nested data structure. The second approach is to analyze individual participant data separately and sequentially for each study and combine them across studies using meta-analysis models (typically called a two-step IPD meta-analysis approach). The third approach is to analyze individual participant data in one simultaneous analysis (also called a one-step individual participant data meta-analysis or IDA). Across all approaches, it is required the interpretation of results should be equivalent across studies, which is a major challenge due to between-study heterogeneity. We provide illustrative examples of the innovative statistical analyses that we have undertaken to make sure that resulting data interpretation is valid. We have conducted simulation studies to make sure our computing codes correctly recover true parameters. Furthermore, we have also conducted sensitivity analysis, such as leaving one study out at a time and repeating analysis, to assess how robust findings are.

Conclusions: The use of individual participant data in meta-analysis presents major opportunities for new discoveries and evidence based practices.