Session: How Can Automatic Text Mining Methods Improve Systematic, Structural, Integrative, and Meta-Analytic Reviews? (Society for Prevention Research 24th Annual Meeting)

Systematic, structured, integrated, and meta-analytic reviews are critical to advancing prevention science and for applying research to practice (Higgins & Green, 2008).  Such reviews are critical to advancing prevention science as they provide a comprehensive view of the state-of-the-art on a particular topic through the identification and synthesis of data, whether it be to determine the average effect of prevention programs for a particular condition (meta-analysis) or evaluate the research evidence for a particular research question (systematic, structured, or integrative review).  Yet, reviews are labor intensive to conduct due to the careful examination of the potentially vast number of publications that is currently completed by human coders.  Fortunately, machine learning and automatic text mining methodologies can simplify the process of searching, summarizing, extracting, and reporting certain types of reviews. Machine learning methods have been suggested for this purpose by the Institute of Medicine (2015) but have yet to be developed or evaluated.  In this symposium, we present how these technologies can be applied to three studies for very different purposes and at different stages in the review process from initial screening of keywords and phrases to the identification of populations and measures of interest within the article texts. When applicable, these automatic methods are compared with human coding in order to assess the performance of these automatic methods as well as the potential cost savings they afford.

In the first paper, automatic text processing is used to screen the titles and abstracts resulting from a structured review of the literature focused on the amplifying factors that contribute to the development pediatric obesity (N = 933), which was conducted to inform preventive intervention strategies. The results were compared to human coding.

In the second paper, we present a set of automatic text processing tools that extend traditional keyword search with semantic, sentence level searches in order to identify relevant papers that include LGBT in its sample population.

The third paper aimed to identify the availability of suicide-related data from 249 adolescent depression prevention trials. A text-mining program was developed to identify studies that administered measures that contained items or subscales related to suicidality.

The high accuracy rates and significant time savings compared to human coding indicate the promise of these methods for a wide variety of applications relevant to prevention science. The symposium Discussant will speak to the potential public health impact of these methods.