Development of a New Method to Characterize the Portfolio of NIH Primary and Secondary Prevention Research in Humans

Introduction: In the past, the Office of Disease Prevention (ODP) has relied on the Research, Condition, and Disease Categorization (RCDC) system to identify prevention research. However, the RCDC system does not provide sufficient detail to classify studies. Therefore, the ODP recently developed a new method to enable a detailed and standardized characterization of prevention grants funded by the National Institutes of Health (NIH). The ODP tested the RCDC method, the ODP method, and a combined approach utilizing both methods to see how they performed at identifying primary and secondary prevention research in humans.

Methods: To characterize prevention research grants, the ODP developed a team coding process that utilized a prevention research taxonomy. The taxonomy contained six categories: study rationale, independent variables, dependent variables, population, study design, and type of prevention research. A team of three research analysts read the title, abstract, and public health relevance statement of each grant and individually coded each grant according to the ODP protocol. Then, the analysts discussed their individual coding and developed a consensus. The ODP staff reviewed 10% of all coded grants for quality control using the same process, and finished with a final consensus. Machine learning algorithms were trained and used to identify prevention research. The ODP estimated the sensitivity, specificity, positive predictive value, and F1 score of the RCDC method, the ODP’s method, and a combined approach that used both methods.

Results: We will show preliminary results on the accuracy of the RCDC method, the ODP method, and a combined method in identifying prevention grants.

Conclusions: The ODP method to systematically characterize NIH prevention research grants will enable detailed analyses of this portfolio to identify trends and gaps in prevention research. This new approach may also be applicable to other broad scientific topics funded by NIH.