Improving Validity of Physical Activity Measurement: An Application of the Two Method Measurement Model

Accurate measurement of physical activity (PA) is an essential component of PA-related prevention program evaluation. PA is measured predominantly via self-report (SR) and frequently by the IPAQ. Accelerometer (ACC) use is rising but intervention effect sizes tend to be weaker or non-significant than those found with SRs.

In theory, it is possible to know exactly how much time was spent being physically active despite a person’s ability to recall this information, or influence by other desires. Thus, expensive ACCs are often used as objective measures to limit the potential for SR bias. For situations where a gold-standard measure is limiting due to expense, and cheaper measures are limiting due to questionable validity, Two-Method Measurement (TMM; Graham et al., 2006) is an effective solution.

TMM improves confidence in the validity of the measurement of one construct by drawing strength from two forms of measurement of the same construct. The model works best when one measure is the gold-standard measure of a construct. Unfortunately, ACCs are not a gold standard; they measure only some activities well, and participants often fail to wear ACCs as instructed.

Thus SRPA-ACC correlations are typically low and the standard TMM model is not the correct model. We present a modified TMM strategy for analyzing PA data. We identify half the participants for whom the SRPA-ACC correlation is high, and for whom the standard TMM model is the correct model. The logic is that when the correlation between two measures is high we infer both measures are measuring the same construct. All data for the group with a high SRPA-ACC correlation are used with the standard TMM model; ACC data are set to missing for the other participants.

We present results from two empirical data sets demonstrating the value of our method. Study 1 uses this method to reassess the intention-behavior gap for PA in a sample of 128 college students with two weeks of daily IPAQ and accelerometer data. We show that using only one measure of PA yields significant bias, and that the TMM strategy produces more reasonable results, and an even larger intention-behavior gap.

Study 2 demonstrates how this same method is also effective in an urban sample of 554 middle-school students where the SR instrument, ACC device, and data collection strategy were all different from those in Study 1.  Variables studied include PA attitudes, physical conditioning, and PA goals. We further show how using just SRPA or ACCs produces bias; and using our modified TMM approach yields reasonable results.

Our method offers prevention scientists a valuable and accessible approach to measuring PA and evaluating PA interventions; in addition to being a measurement tool capable of expansion to other applications.