Findings: The first pilot program conducted in 2014 included approximately 1,500 Soldiers from three participating units, who were asked to complete an online survey at two time points. These newly collected survey data were integrated with existing administrative data (e.g., physical fitness test scores) and other data sources (e.g., personal fitness tracking data). Results from evaluation of the 2014 program did not demonstrate program effectiveness in changing Soldiers’ SAN behaviors, however, it was unclear whether this was due to the program or methodological issues (e.g., sample size, data quality, evaluation design). Several modifications to the program delivery and design were recommended to improve the program’s potential effectiveness, and evaluation rigor. These included increasing the participant number, including a comparison condition, having on-site program support, and adding midpoint data collection.
Many recommended changes were executed in a subsequent 2015-2016 pilot program. The 2015-2016 program expanded to include over 14,000 Soldiers from 37 units assigned to either a program or comparison condition. Participating Soldiers completed an online survey at three time points (i.e., baseline, midpoint, endpoint). These data were matched and then integrated with existing administrative data. Findings from this second program offered key evidence and feedback to continue to inform P3 Initiative efforts. Continued evaluation analyses of P3 inform and shape health intervention efforts.
Conclusions: Lessons learned from the evaluation of this health intervention program shed light on successful strategies for program development and implementation, big data integration, using evidence to inform program decision-makers, and dissemination of evaluation evidence. Coordinating and integrating multiple data sources, including newly collected survey data and existing military datasets, can be beneficial for formative and summative program evaluation efforts.