Using Big Data to Inform an Army Behavioral Health Intervention Program

Introduction: The U.S. Army is committed to supporting healthy Soldiers and enhancing healthy sleep, activity, and nutrition, referred to as the Performance Triad (P3). The P3 Initiative aims to improve these behaviors to decrease musculoskeletal injury and increase overall health and readiness. Two pilot programs have been conducted and evaluated, integrating evaluation data and multiple, large, administrative datasets. This paper presents these evaluation findings and the opportunities and challenges of integrating big data into evaluations that inform program decision-makers.

Findings: The first pilot program conducted in 2014 included approximately 1,500 Soldiers from three participating units, who were asked to complete an online survey at two time points. These newly collected survey data were integrated with existing administrative data (e.g., physical fitness test scores) and other data sources (e.g., personal fitness tracking data). Results from evaluation of the 2014 program did not demonstrate program effectiveness in changing Soldiers’ SAN behaviors, however, it was unclear whether this was due to the program or methodological issues (e.g., sample size, data quality, evaluation design). Several modifications to the program delivery and design were recommended to improve the program’s potential effectiveness, and evaluation rigor. These included increasing the participant number, including a comparison condition, having on-site program support, and adding midpoint data collection.

Many recommended changes were executed in a subsequent 2015-2016 pilot program. The 2015-2016 program expanded to include over 14,000 Soldiers from 37 units assigned to either a program or comparison condition. Participating Soldiers completed an online survey at three time points (i.e., baseline, midpoint, endpoint). These data were matched and then integrated with existing administrative data. Findings from this second program offered key evidence and feedback to continue to inform P3 Initiative efforts. Continued evaluation analyses of P3 inform and shape health intervention efforts.

Conclusions: Lessons learned from the evaluation of this health intervention program shed light on successful strategies for program development and implementation, big data integration, using evidence to inform program decision-makers, and dissemination of evaluation evidence. Coordinating and integrating multiple data sources, including newly collected survey data and existing military datasets, can be beneficial for formative and summative program evaluation efforts.