Skip Navigation


American Journal of Epidemiology Advance Access originally published online on August 1, 2007
American Journal of Epidemiology 2007 166(7):832-840; doi:10.1093/aje/kwm148
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
166/7/832    most recent
kwm148v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (17)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Ferrari, P.
Right arrow Articles by Matthews, C. E.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ferrari, P.
Right arrow Articles by Matthews, C. E.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

American Journal of Epidemiology © The Author 2007. Published by the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

PRACTICE OF EPIDEMIOLOGY

The Role of Measurement Error in Estimating Levels of Physical Activity

Pietro Ferrari1, Christine Friedenreich2 and Charles E. Matthews3

1 Nutrition and Hormones Group, International Agency for Research on Cancer, Lyon, France
2 Division of Population Health and Information, Alberta Cancer Board, Calgary, Alberta, Canada
3 Division of General Internal Medicine, Department of Medicine, Vanderbilt University School of Medicine, Nashville, TN

Correspondence to Dr. Pietro Ferrari, Nutrition and Hormones Group, International Agency for Research on Cancer, 150 Cours Albert Thomas 69372 Lyon, France (e-mail: ferrari{at}iarc.fr; www.iarc.fr).

Received for publication November 15, 2006. Accepted for publication April 9, 2007.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Epidemiologic studies have demonstrated that physical inactivity is an important determinant of numerous chronic diseases. However, self-reported estimates of physical activity contain measurement errors responsible for attenuating relative risk estimates. A validation study conducted in 2002–2003 at the Alberta Cancer Board (Canada) included a physical activity questionnaire, four 7-day physical activity logs, and four sets of accelerometer data from 154 study subjects (51% women) aged 35–65 years. The authors used a measurement error model to evaluate validity of the different types of physical activity assessment, and the attenuation factors, after taking into account error correlations between self-reported measurements. The validity coefficients, which express the correlation between measured and true exposure, were higher for accelerometers (0.81, 95% confidence interval (CI): 0.76, 0.85) compared with the physical activity log (0.57, 95% CI: 0.47, 0.66) and questionnaire measurements (0.26, 95% CI: 0.12, 0.40). The estimate of the attenuation factor for questionnaires was 0.13 (95% CI: 0.05, 0.23). Accuracy of physical activity questionnaire measurements was higher for men than for women, for younger individuals, and for those with a lower body mass index. Because the degree of attenuation in relative risk estimates is substantial, after the role of error correlations was considered, validation studies quantifying the impact of measurement errors on physical activity estimates are essential to evaluate the impact of physical inactivity on health.

bias (epidemiology); measurement error; questionnaires

Abbreviations: CI, confidence interval; PA LOG, physical activity log; PAQ, physical activity questionnaire


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
It is recognized that physical activity measurements used in large epidemiologic investigations contain a sizable level of measurement error (1–3). Under certain conditions, when measurement errors are nondifferentially expressed, the association between physical activity and an outcome of interest is attenuated toward the null hypothesis of no association (4–7), thus making it difficult to evaluate the likely beneficial effects of physical activity on the full range of health outcomes.

It is possible to estimate the magnitude of measurement errors and the impact on the estimation of associations when assessing individuals' physical activity exposure. In so doing, the effect of measurement errors can be quantified. Numerous statistical methods to correct for bias attributable to measurement error when evaluating a statistical association between an exposure and an outcome of interest have been proposed (8–10).

No "gold standard" measurements of physical activity levels among free-living individuals are available to directly measure true levels of exposure. Physical activity assessment methods that make use of recall methods, such as questionnaires, are a subjective means of estimating individual exposure because they rely on individuals' ability to remember levels of exposure. It has been observed that many factors influence the accuracy of exposure estimates, all based primarily on personal perception of the exposure being evaluated and quantified.

Design and analysis of validation studies in terms of statistical latent variable models (11, 12) have shown that the validity coefficient can be estimated by comparison with a third objective measurement. It is assumed that errors in the objective measurements are independent from self-reported measurements.

Currently, the methods that provide the most feasible and accurate measurements of activity in large validation studies are physical activity logs (PA LOGs) and objective measures obtained from accelerometers. PA LOGs are thought to minimize errors associated with long-term recall of activity and estimation of activity frequency, whereas objective measures do not rely on self-report at all. When assessing the validity of physical activity measurements, it is therefore important to use statistical models that account for random and systematic errors, as well as error correlation between questionnaires and PA LOG measurements and between replicates of the same instrument.

In a recent study (13), validity and reliability of a self-administered past-year physical activity questionnaire (PAQ) developed for use in research studies conducted at the Alberta Cancer Board (Canada) were assessed by comparison with 7-day PA LOGs and 7-day accelerometer data. In the absence of individual estimates of true physical activity level, 7-day PA LOG and accelerometer measurements were used as the reference methods. In this paper, we propose to acknowledge and take into account measurement errors in all observed measurements and to estimate the validity of PAQs, PA LOGs, and accelerometer data. This assessment is accomplished by means of structural equation models, where inclusion of objective measures of physical activity allows estimation of correlation between errors of self-report instruments (questionnaire and PA LOGs). In addition, we estimate the attenuation factor that quantifies the effect of measurement error on the association with an outcome. Numerous aspects of measurement error in individual estimates of physical activity level are evaluated and discussed.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 APPENDIX
 References
 
Data were available on 154 men and women aged 35–65 years who completed four PAQ measurements (PAQ1–PAQ4), four 7-day PA LOGs, and four sets of accelerometer data over a 1-year period. The design and methods for this initial validation study have been described previously (13).

Assessment of physical activity
In brief, two measurements of physical activity in the previous 12 months were taken at baseline in a 1-year follow-up study by using the self-administered PAQ known as the Past Year Total Physical Activity Questionnaire. These repeat measurements were taken, on average, 9 weeks apart. At the end of the 1-year follow-up validation study, the questionnaire was completed again, first as a self-administered (PAQ3) and then as a telephone-administered (PAQ4) questionnaire. During the year-long follow-up, study participants wore the accelerometers for four 1-week periods at intervals approximately 12 weeks apart. Immediately after wearing the accelerometers for 1 week, they completed the 7-day PA LOGs by coding their activity for each 15-minute interval of every hour of the day for 7 days consecutively. These data were transformed to express metabolic equivalent task-hours per week, as determined from the Compendium of Physical Activities (14, 15).

Hence, during the year, four sets of accelerometer data and PA LOGs were collected. Accelerometer data estimate the intensity and duration of motion in the sampling interval (i.e., counts/minute). The basic activity count data were summarized in terms of hours per week and, using total energy expenditure (metabolic equivalent task-hours/week), were derived from the equation of Swartz et al. (16), after censoring recorded values below 150 counts/minute. In addition, physical activity that was done when the accelerometers were not worn was recorded, with a separate activity monitor log that the respondents completed.

Statistical model
In the present study, the validity of PAQ3, accelerometer, and PA LOG measurements was evaluated. This evaluation was achieved by estimating the validity coefficient that reflects the correlation between the observed measurement of exposure, be it questionnaire, accelerometer data, or PA LOG, and the unknown true level (T). A measurement error model was therefore defined that assumes relations between questionnaire (Q), PA LOG (R), and accelerometer (A) measurements and true level of activity, for subject i = 1, ..., I, and measurements j = 1, ..., JX (X = R, A), as

Formula (1a)

Formula (1b)

Formula (1c)
The following relations are assumed for random measurement errors {varepsilon}Qi and {varepsilon}Xij, E({varepsilon}Qi|Ti) = 0, E({varepsilon}Rij|Ti) = 0, E({varepsilon}Aij|Ti) = 0, and Var({varepsilon}Qi) = {sigma}Formula, Var({varepsilon}Rij) = {sigma}Formula, and Var({varepsilon}Aij) = {sigma}Formula. For the questionnaire measurement Q, the coefficients {alpha}Q and ßQ in model 1a express constant and proportional scaling biases (17), while the residual terms {varepsilon}Qi models the random part of measurement errors in Q and are assumed to be uncorrelated with true physical activity level Ti, after {alpha}Q and ßQ have captured the systematic component of measurement error (12).

In this analysis, the 7-day PA LOG was chosen to be the "reference measurement," with {alpha}R = 0 and ßR = 1. The relation expressed in model 1b assumes that errors are strictly random and that variation around individuals' unknown true physical activity is entirely attributable to within-person random variability or to random measurement errors in reporting physical activity levels (i.e., cov({varepsilon}Rij, Ti) = 0, for {forall} j isin JR).

In addition, throughout this analysis, it is assumed that random errors in questionnaire and 7-day PA LOG measurements are correlated (cov({varepsilon}Qi,{varepsilon}Rij) != 0, for {forall} j isin JR), as a result of individuals' tendency to consistently misreport their activity level. Furthermore, correlation between errors in questionnaire-based assessments may arise because these measurements may share some common source of variability. Similar assumptions were made for the replicates of PA LOG measurements (cov({varepsilon}Rij,{varepsilon}Rik) != 0; j != k).

Although measurements collected by using an accelerometer have been reported to be reliable for estimating dynamic physical activities (18–22), some concerns have been raised about the capacity of accelerometers to provide a valid estimate of absolute physical activity levels (20). For this reason, in expression 1c, the terms {alpha}A and ßA were introduced for accelerometer data to take into account systematic error, while {varepsilon}A captures the random measurement error component.

Accelerometer measurements provide objective estimates of physical activity levels. It was assumed that random errors in questionnaire-based and accelerometer measurements are independent, that is, cov({varepsilon}Qi,{varepsilon}Aij) = cov({varepsilon}Rij,{varepsilon}Aij) = 0 for {forall} j isin JA and {forall} j isin JR, under the hypothesis that errors in the latter are not influenced by study subjects' ability to recall past activity levels. Measurement errors in model 1 could also be expressed in terms of person-specific bias and random variation (23, 24). However, the two notations are equivalent, as has been shown previously (25).

Model parameter estimates can be obtained through structural equation models. In this context, given the linear associations in expression 1 and the related assumptions regarding error variances and covariances, the population covariance matrix ({Sigma}) of a set of observed variables (X1, ..., Xp) is a function of a set of basic parameters (q = (q1, ..., qt)), as Formula = {Sigma}(Formula ) (26, 27). For the model to be identifiable, a situation that arises when a unique set of parameter estimates corresponds to the observed data, it is necessary that the components of q are not more than the elements of the sample covariance matrix S, that is, t ≤ p(p + 1)/2. Classically, the maximum likelihood method is used for such model fitting, which usually requires the assumption that variables are normally distributed (27). Notably, in this study, the components of q are the regression coefficients in equation 1 as well as the variance of the independent variable ({sigma}T2), and the error variances and covariances.

Having estimated the parameter vector q, the validity coefficient for questionnaire Q can be calculated, as detailed in the Appendix:

Formula (2)
The validity coefficients for the accelerometer measurements, {rho}AT, and PA LOG, {rho}RT, are estimated in the same manner.

When measurement errors in exposure variables are present, the estimated association describing the exposure-disease relation (e.g., relative risk estimates) may be biased, typically toward the null. A number of statistical methods have been proposed for correcting the effects of covariate measurement error (7–9). One simple adjustment for measurement error is the regression calibration method (8), where the attenuation factor, {lambda}, is estimated by the slope of true on observed exposure in a validation study. The log-relative risk in the disease model is corrected (deattenuated) by dividing by Formula (8). For this reason, values of the attenuation factor close to 0 lead to more serious underestimations of risk. In this work, the attenuation factor is calculated as

Formula (3)
The regression calibration method corrects for random and systematic within-person error in observed measurements of exposure, provided that a reference measure is available (24). By estimating the validity coefficient and the attenuation factor using models 1 and 2, the possible error correlation between questionnaire and PA LOG measurements is quantified and the effect of taking into account this error correlation is evaluated.

A model with an instrumental variable
It is conceivable that replicates of accelerometers contain components of intraindividual systematic errors, because accelerometer data for subjects who engage in very dynamic kinds of pursuits (e.g., walking, running) for a larger proportion of their overall activity would be predicted to have less systematic error, whereas the actigraph data for participants who engage in more static pursuits for a larger proportion of their activity, which may expend considerable energy but cause less acceleration of the body (e.g., household chores, gardening), might contain systematic errors. For this reason, two strategies have been used to examine these possible errors.

Estimation of error correlation in replicates of accelerometer data in models 1a–1c leads to identifiability issues. To address this, two complementing strategies have been followed. First, a sensitivity analysis was conducted in model 1 by imputing values of the correlation between errors in replicates of accelerometer data, thus pursuing estimation of the validity coefficients. Second, an additional model was considered by introducing an instrumental variable (24, 28, 29), as follows:

Formula (4)
where Ti, {alpha}Q, {alpha}A, ßQ, ßA, {varepsilon}Qi, {varepsilon}Rij, and {varepsilon}Aij have been defined in model 1. It is further assumed that the errors in Bi measurements are E({varepsilon}Bi|Ti) = 0, Var({varepsilon}Bi) = {sigma}{varepsilon}B2 and are uncorrelated with any other residual error terms in model 4. Furthermore, it is assumed that (cov({varepsilon}Aij,{varepsilon}Aik) != 0; {forall} j isin JA, with j != k). Parameter estimation was conducted by using structural equation models, similarly to model 1. Details are provided in the Appendix.

Structural equation models assume that the random variables approximate a multivariate normal distribution. For this reason, the data have been log transformed. To deal with missing values (n = 56 over the nine different physical activity variables), an expectation-maximization algorithm was used to estimate the maximum likelihood covariance matrix that was used as input data to estimate model parameters in the structural equation model. This approach assumes that data are missing at random (30). Parameter estimates were obtained by using the CALIS procedure in SAS software (31), as well as associated approximate standard errors. For validity coefficient and attenuation factor estimates, 95 percent confidence intervals were obtained by computing the 2.5th and 97.5th percentiles of a distribution determined with bootstrap sampling (32). A total of 1,000 repetitions gave sufficiently stable intervals.

Models for men and women combined were systematically adjusted for gender by regressing physical activity measurements on gender and using residuals. To evaluate the degree of correlated errors after controlling for study subjects' characteristics, residuals of physical activity measurements on, in turn, age, body mass index, and level of physical fitness at baseline were computed. However, results were very similar, and only unadjusted results are presented here.

In addition, in an attempt to evaluate the effect of measurement errors on the validity of physical activity measurements, models were run according to study subjects' age, body mass index, and level of physical fitness at baseline. Separately for each variable, two groups were created by taking, respectively, cutoffs equal to 50 years for age, the gender-specific body mass index median values, equal to 27.5 and 25.7 (kg/m2) in men and women, respectively, and the gender-specific level of fitness, based on predicted maximum oxygen consumption at baseline, equal to 31.3 and 25.8 (ml/kg per minute) in men and women, respectively.

Log-transformed body weight was used to estimate the validity of questionnaire measurements using an instrumental variable in model 4. Using level of fitness instead of weight provided very similar results.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 APPENDIX
 References
 
Although average values in log-transformed questionnaire, PA LOG, and accelerometer physical activity measurements were not very different, greater heterogeneity was observed for variability in the different types of data (table 1). Mean values were very similar for men and women, but variability of different measurements was higher in men than in women.


View this table:
[in this window]
[in a new window]

  		TABLE 1. Overall and gender-specific sample size, mean, and standard deviation of log-transformed PAQ* and 7-day PA LOG,* ACC* measurements (log-MET*-hours/week), and weight (kg)

 
The values for estimates of the validity coefficients were higher for accelerometer measurements (0.81, 95 percent confidence interval (CI): 0.76, 0.85) compared with PA LOG (0.57, 95 percent CI: 0.47, 0.66) and questionnaire (0.26, 95 percent CI: 0.12, 0.40) measurements (table 2). Higher validity coefficients for questionnaire measurements were observed for men (0.39, 95 percent CI: 0.20, 0.58) than for women (0.15, 95 percent CI: 0.01, 0.33), while similar coefficients were observed for PA LOG and accelerometer data for the two genders. The 95 percent confidence intervals showed that these parameters were estimated with precision. Attenuation factor estimates for questionnaires were equal to 0.13 (95 percent CI: 0.05, 0.23), with a lower value for women (0.07, 95 percent CI: –0.03, 0.18) than for men (0.23, 95 percent CI: 0.09, 0.41). True variability was equal to 0.07 (95 percent CI: 0.04, 0.10).


View this table:
[in this window]
[in a new window]

  		TABLE 2. Overall (n = 154) and sex-specific (men, n = 75; women, n = 79) estimates of validity coefficient (Table 2XT) and attenuation factors (Table 2), estimates of true variance (Table 2Formula), error correlations (Table 2{varepsilon}ij{varepsilon}ik), and associated 95% confidence intervals under model 1 for measurements X, in turn, PAQ,* 7-day PA LOG,* and ACC* measurements

 
Higher error correlations were observed between replicates of PA LOG measurements (0.55, 95 percent CI: 0.45, 0.64) than between questionnaire and PA LOG (0.23, 95 percent CI: 0.07, 0.38). Similar values were observed for men and women.

The results of assessment of measurement error by study subjects' specific characteristics (table 3) show that validity of questionnaire measurements varied by age, with lower coefficients observed for subjects older than age 50 years, and also by body mass index, with lower validity for participants with a higher body mass index. Although adjustment for age did not produce noticeable changes, some minor confounding by age cannot be ruled out. Similarly, higher coefficients were observed for individuals with a better level of fitness, although the lower body mass index values and, to a lesser extent, the younger age of participants with higher predicted maximum oxygen consumption might have partially driven these findings.


View this table:
[in this window]
[in a new window]

  		TABLE 3. Age, BMI,* and level-of-fitness group-specific means and estimates of gender-adjusted validity coefficients (Table 3XT), and associated 95% confidence interval under model 1, for measurements X, in turn, PAQ,* 7-day PA LOG,* and ACC* measurements

 
The sensitivity analysis showed that the validity coefficients and the attenuation factors increased slightly with increasing values of the correlation between errors in replicates of accelerometer measurements, although only for values of accelerometer error correlations higher than 0.4 was the change (bias) appreciable (table 4).


View this table:
[in this window]
[in a new window]

  		TABLE 4. Values of the validity coefficients for questionnaire measurements (Table 4QT), attenuation factor (Table 4), correlation between error in questionnaire and PA LOG (Table 4{varepsilon}Rij,{varepsilon}Qik), and correlation between errors in replicates of PA LOG (Table 4{varepsilon}Rij,{varepsilon}Rik) for different values of the correlation between error in replicates of accelerometer data ({rho}{varepsilon}Aij,{varepsilon}Aik), as estimated in a sensitivity analysis

 
When an instrumental variable was used, the questionnaire validity coefficient was equal to 0.27 (95 percent CI: 0.13, 0.49), a value very similar to the coefficient estimated in model 1 (table 5). The correlation between error in replicates of accelerometer measurements was equal to 0.17 (95 percent CI: –0.75, 0.54). The confidence intervals for correlation between errors in model 4 were wide and asymmetrical.


View this table:
[in this window]
[in a new window]

  		TABLE 5. Estimates of validity coefficient (Table 5XT) and attenuation factors (Table 5) for measurements X, in turn, PAQ,* 7-day PA LOG,* and ACC* measurements, estimates of true variance (Table 5Formula), and error correlations (Table 5{varepsilon}ij{varepsilon}ik) and associated 95% confidence interval under model 4, assuming correlation between errors in replicates of accelerometer measurements

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
APPENDIX
 References
 
We evaluated the validity of PAQ, PA LOG, and accelerometer measurements by using an integrated structural equation model. It was assumed that none of these measurements are themselves true gold standards but rather contain components of random and systematic measurement error. Although earlier studies investigated the role of measurement error in physical activity assessments (33, 34), to our knowledge we evaluated and quantified for the first time in this work the effect of error correlations in self-reported physical activity measurements.

The models used in this study relied on several statistical assumptions. First, it was assumed that PA LOG measurements can be used as the reference and that errors are strictly random and unrelated to true long-term physical activity level (3, 35, 36). It should be noted that PA LOGs are burdensome for participants to complete, and they measure activities specified a priori, thus having the potential to introduce systematic errors. However, if the assumption of random error in PA LOG measurements does not hold (i.e., model 1b), underestimation of the validity coefficients can result. Therefore, our estimates are likely to be conservative. Second, to estimate all the parameters, thus making the model identifiable, a third objective measurement is needed whose errors are assumed to be independent of the self-reported physical activity measurements (11, 12, 24, 37).

For this purpose, accelerometer data were used, which were assumed to contain random and systematic measurement errors. Accelerometers optimally capture more dynamic activities (e.g., walking and running) but provide limited or no estimates of cycling or swimming, which in this study were systematically recorded with monitor logs. Although this could introduce an aspect of self-report to the actigraph, the contribution of these activities to total activity is likely very low. We found the validity coefficients of PAQ measurements to be threefold lower for women, and of similar magnitude for men, compared with values previously observed (13). This result is mainly attributable to the fact that the methods used in this study allow the error correlation between replicates of the same types of instruments, as well as between different types, to be evaluated and taken into account. Estimating the validity coefficients by comparison of measurements that share the same source of errors can lead to overestimating the quantity of interest (23).

It is realistic to assume that the sources of errors in self-reported estimates of physical activity are similar, thus leading to error correlations between the measurements of two types of instruments—in our study, PAQ and PA LOGs—and between replicates of the same instrument. The results of this study confirm that, after the role of error correlations has been taken into account, the validity coefficients of physical activity levels from questionnaire measurements can be relatively low. It is therefore of primary interest to set up validation studies in which physical activity levels are estimated through different assessment instruments.

In this study, the assumption that errors in replicates of accelerometer measurements are uncorrelated was challenged through a sensitivity analysis and by considering a model with an instrumental variable (24, 28, 29). Our results provide some evidence that the impact of error correlation in replicates of accelerometers is minor, as the validity coefficient is not sizably affected for values of accelerometer error correlations ranging from –0.4 to 0.4. To estimate the validity coefficient, it was assumed that individual weights were linearly related to the true level of the log of physical activity and that errors in this relation were uncorrelated with errors in physical activity measurements (Q, R, and M). Concerns regarding the appropriateness of these assumptions may exist, and our results need to be replicated in models with objective measurements with independent errors over replicates (e.g., doubly labeled water).

We also evaluated the effects of measurement errors on the validity of questionnaire measurements according to study subjects' characteristics, notably age, body mass index, and level of fitness. Higher validity coefficients were observed for younger subjects (below age 50 years) and for subjects below the gender-specific body mass index medians, thus suggesting that these factors are associated with accuracy of estimates of physical activity level. These results provide evidence that measurement errors in the assessment of past year physical activity patterns have a strong systematic component, which can be the result of within-person systematic error not randomly distributed between subjects (38). Kipnis et al. (23) refer to this bias as person- and group-specific bias. Although the effect of body mass index on the validity coefficients of dietary measurements has been repeatedly suggested in the dietary assessment literature (39–41), the evidence on the role of study subjects' characteristics in the accuracy of physical activity measurements has been more mixed, with some studies demonstrating an effect of body mass index on reporting errors (42, 43), while others have not (3, 44). However, concerns could be raised about whether these results could be generalized to populations different from the one under study, that is, with a different age structure or ethnic composition.

We have shown that the degree of attenuation of risk estimates derived from instruments such as the PAQ is likely substantial. Thus, the magnitude of the associations between physical activity and health outcomes, such as cancer or cardiovascular disease, may be severely attenuated by measurement errors. We estimated an attenuation factor equal to 0.13 for the PAQ, and, given this level of attenuation, a true relative risk of 2 would be estimated as a relative risk of 1.10. Therefore, to estimate the magnitude of the association between physical activity and health more accurately, carefully conducted calibration studies may be required, in which different types of physical activity assessment are used, possibly on a subsample of a large-scale epidemiologic study.

Our findings are based on log-transformed variables and therefore apply to risk models in which log of physical activity measurements is related to a specific disease outcome. However, results on untransformed variables provided similar results (data not shown). Results from this study also have implications for the design of calibration studies. The correlation between errors in self-reported measurements was substantial and must be properly accounted for in the measurement error model; otherwise, the model might only partially correct for the effect of measurement error. In our study, estimates of attenuation factors using a measurement error model were generally two- to threefold lower than attenuation estimated in a "standard" regression calibration model, in which questionnaire measurements were directly regressed on PA LOG values, without using accelerometer data. The attenuation factor estimated in this way was equal to 0.30 (data not shown), thus leading to underestimation of the deattenuated risk, which would be equal to 1.23 for a true relative risk of 2. These results suggest that the overall level of attenuation may be greater than previously expected, which confirms the importance of estimating the attenuation factors accurately because quantification of the impact of physical activity on public health outcomes will be directly affected by such attenuation.

In the absence of objective measurements that provide accurate estimates of absolute level of physical activity, questionnaires should be combined with accelerometer data and PA LOG measurements. Accelerometers ensure independence of errors with self-reported measurements, thus fulfilling fundamental requirements for the identifiability of statistical models for complex validation study designs. In this way, it is possible to estimate the various quantities that compose the error structure and to better understand the associations between physical activity and health.

The design of validation studies therefore requires great care, as well as thoughtful considerations concerning the sample size of the study, to enable the potential sources of errors, attributable to study subjects' individual characteristics and/or particular types of physical activity, to be investigated in depth.


   APPENDIX
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
References
 
In this work, parameter estimates were obtained through the CALIS procedure (31) that uses the maximum likelihood estimation algorithm. As described in the text, one questionnaire, four PA LOGs, and four accelerometer measurements were modeled in expression 1. In this study, six linear constraints were used for error correlation among replicates of PA LOG measurements, as {sigma}{varepsilon}Rij,{varepsilon}Rik = {sigma}{varepsilon}R,{varepsilon}R for {forall} j, k isin JR with j != k, as well as four constraints for the error correlation between questionnaire and PA LOG, as {sigma}{varepsilon}Qi,{varepsilon}Rij = {sigma}{varepsilon}Q,{varepsilon}R for {forall} j isin JR. Parameter estimates can be alternatively obtained with the method-of-moments estimators, as shown by Spiegelman et al. (24). In model 1, there are eight second moments that allow eight model parameters to be uniquely determined, as

Formula

Following the above associations, the validity coefficient for questionnaire measurements that estimates the correlation between questionnaire Q and true physical activity level, T, is equal to

Formula
Similarly, for the attenuation factor,

Formula
In model 4, an instrumental variable has been added to the model. It is assumed that {sigma}{varepsilon}Aij,{varepsilon}Aik = {sigma}{varepsilon}A,{varepsilon}A for {forall} j,k isin JA with j != k. Similarly to model 1,

Formula

In this model, the number of moments (n = 12) exceeds the number of parameters (n = 11). Therefore, more than one estimator is available for the parameters ßQ, {sigma}{varepsilon}Q,{varepsilon}R, and {sigma}Formula. In this work, model parameters are estimated iteratively by a nonlinear optimization algorithm in the CALIS procedure (31), which uses maximum likelihood theory.


   ACKNOWLEDGMENTS
 
The authors thank Heather Neilson for reviewing drafts of the manuscript. They also thank Bernard Rosner for his helpful suggestions regarding formalization of the statistical model.

Conflict of interest: none declared.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 

  1. Philippaerts RM, Westerterp KR, Lefevre J. Comparison of two questionnaires with a tri-axial accelerometer to assess physical activity patterns. Int J Sports Med (2001) 22:34–9.[CrossRef][Web of Science][Medline]
  2. Schuit AJ, Schouten EG, Westerterp KR, et al. Validity of the Physical Activity Scale for the Elderly (PASE): according to energy expenditure assessed by the doubly labeled water method. J Clin Epidemiol (1997) 50:541–6.[CrossRef][Web of Science][Medline]
  3. Adams SA, Matthews CE, Ebbeling CB, et al. The effect of social desirability and social approval on self-reports of physical activity. Am J Epidemiol (2005) 161:389–98.[Abstract/Free Full Text]
  4. de Klerk NH, English DR, Armstrong BK. A review of the effects of random measurement error on relative risk estimates in epidemiological studies. Int J Epidemiol (1989) 18:705–12.[Abstract/Free Full Text]
  5. Armstrong BG. The effects of measurement errors on relative risk regressions. Am J Epidemiol (1990) 132:1176–84.[Abstract/Free Full Text]
  6. Fuller WA. Measurement error model. (1987) New York, NY: Wiley.
  7. Carroll R, Ruppert D, Stefansky LA. Measurement error in nonlinear models. (1995) London, United Kingdom: Chapman & Hall.
  8. Rosner B, Willett WC, Spiegelman D. Correction of logistic regression relative risk estimates and confidence intervals for systematic within-person measurement error. Stat Med (1989) 8:1051–69.[Web of Science][Medline]
  9. Armstrong BG, Whittemore AS, Howe GR. Analysis of case-control data with covariate measurement error: application to diet and colon cancer. Stat Med (1989) 8:1151–63. discussion 1165–6.[Web of Science][Medline]
  10. Prentice RL, Willett WC, Greenwald P, et al. Nutrition and physical activity and chronic disease prevention: research strategies and recommendations. J Natl Cancer Inst (2004) 96:1276–87.[Abstract/Free Full Text]
  11. Plummer M, Clayton D. Measurement error in dietary assessment: an investigation using covariance structure models. Part I. Stat Med (1993) 12:925–35.[Web of Science][Medline]
  12. Kaaks R, Ferrari P, Ciampi A, et al. Uses and limitations of statistical accounting for random error correlations in the validation of dietary questionnaire assessments. Public Health Nutr (2002) 5:969–76.[CrossRef][Web of Science][Medline]
  13. Friedenreich CM, Courneya KS, Neilson HK, et al. Reliability and validity of the Past Year Total Physical Activity Questionnaire. Am J Epidemiol (2006) 163:959–70.[Abstract/Free Full Text]
  14. Ainsworth BE, Haskell WL, Leon AS, et al. Compendium of physical activities: classification of energy costs of human physical activities. Med Sci Sports Exerc (1993) 25:71–80.
  15. Ainsworth BE, Haskell WL, Whitt MC, et al. Compendium of physical activities: an update of activity codes and MET intensities. Med Sci Sports Exerc (2000) 32:S498–504.
  16. Swartz AM, Strath SJ, Bassett DR Jr, et al. Estimation of energy expenditure using CSA accelerometers at hip and wrist sites. Med Sci Sports Exerc (2000) 32:S450–6.
  17. Kaaks R, Riboli E, Estève J. Estimating the accuracy of dietary questionnaire assessments: validation in terms of structural equation models. Stat Med (1994) 13:127–42.[Web of Science][Medline]
  18. Freedson PS, Melanson E, Sirard J. Calibration of the Computer Science and Applications, Inc. accelerometer. Med Sci Sports Exerc (1998) 30:777–81.
  19. Janz KF. Validation of the CSA accelerometer for assessing children's physical activity. Med Sci Sports Exerc (1994) 26:369–75.
  20. Matthews CE. Calibration of accelerometer output for adults. Med Sci Sports Exerc (2005) 37(11 suppl):S512–22.
  21. Nichols JF, Morgan CG, Chabot LE, et al. Assessment of physical activity with the Computer Science and Applications, Inc, accelerometer: laboratory versus field validation. Res Q Exerc Sport (2000) 71:36–43.[Web of Science][Medline]
  22. Welk GJ, Schaben JA, Morrow JR Jr. Reliability of accelerometry-based activity monitors: generalizability study. Med Sci Sports Exerc (2004) 36:1637–45.
  23. Kipnis V, Carroll RJ, Freedman LS, et al. Implications of a new dietary measurement error model for estimation of relative risk: application to four calibration studies. Am J Epidemiol (1999) 150:642–51.[Abstract/Free Full Text]
  24. Spiegelman D, Zhao B, Kim J. Correlated errors in biased surrogates: study designs and methods for measurement error correction. Stat Med (2005) 24:1657–82.[CrossRef][Web of Science][Medline]
  25. Michels KB, Day NE. Re: "Implications of a new dietary measurement error model for estimation of relative risk: application to four calibration studies." (Letter). Am J Epidemiol (2000) 152:494–5.[Free Full Text]
  26. Bentler PM, Weeks DG. Linear structural equations with latent variables. Psychometrika (1980) 45:289–308.[CrossRef][Web of Science]
  27. Bollen KA. A new incremental fit index for general structural equation models. Sociol Methods Res (1989) 17:303–16.[Abstract]
  28. Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol (2000) 29:722–9.[Abstract/Free Full Text]
  29. Fraser GE, Butler TL, Shavlik D. Correlations between estimated and true dietary intakes: using two instrumental variables. Ann Epidemiol (2005) 15:509–18.[CrossRef][Web of Science][Medline]
  30. Little RJA, Rubin DB. Statistical analysis with missing data. (1987) New York, NY: Wiley.
  31. SAS Institute, Inc. SAS. Online Doc, version 8. (1999) Cary, NC: SAS Institute, Inc.
  32. Efron B, Tibshirani RJ. An introduction to the bootstrap. (1998) London, United Kingdom: Chapman & Hall.
  33. Wong MY, Day NE, Wareham NJ. Measurement error in epidemiology: the design of validation studies II: bivariate situation. Stat Med (1999) 18:2831–45.[CrossRef][Web of Science][Medline]
  34. Wareham NJ, Wong MY, Hennings S, et al. Quantifying the association between habitual energy expenditure and blood pressure. Int J Epidemiol (2000) 29:655–60.[Abstract/Free Full Text]
  35. Haggarty P, McNeill G, Manneh MK, et al. The influence of exercise on the energy requirements of adult males in the UK. Br J Nutr (1994) 72:799–813.[CrossRef][Web of Science][Medline]
  36. Racette SB, Schoeller DA, Kushner RF. Comparison of heart rate and physical activity recall with doubly labeled water in obese women. Med Sci Sports Exer (1995) 27:126–33.[Web of Science][Medline]
  37. Kipnis V, Subar AF, Midthune D, et al. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol (2003) 158:14–21.[Abstract/Free Full Text]
  38. Willett W. An overview of issues related to the correction of non-differential exposure measurement error in epidemiologic studies. Stat Med (1989) 8:1031–40.[Web of Science][Medline]
  39. Bandini LG, Schoeller DA, Cyr HN, et al. Validity of reported energy intake in obese and nonobese adolescents. Am J Clin Nutr (1990) 52:421–5.[Abstract/Free Full Text]
  40. Heitmann BL. The influence of fatness, weight change, slimming history and other lifestyle variables on diet reporting in Danish men and women aged 35–65 years. Int J Obes (1993) 17:329–36.[Web of Science][Medline]
  41. Heitmann BL, Lissner L. Dietary underreporting by obese individuals—is it specific or non-specific? BMJ (1995) 311:986–9.[Abstract/Free Full Text]
  42. Irwin ML, Ainsworth BE, Conway JM. Estimation of energy expenditure from physical activity measures: determinants of accuracy. Obes Res (2001) 9:517–25.[Web of Science][Medline]
  43. Lichtman SW, Pisarska K, Berman ER, et al. Discrepancy between self-reported and actual caloric intake and exercise in obese subjects. N Engl J Med (1992) 327:1893–8.[Abstract]
  44. Klesges RC, Eck LH, Mellon MW, et al. The accuracy of self-reports of physical activity. Med Sci Sports Exerc (1990) 22:690–7.

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 HypertensionHome page
R. J. van de Laar, I. Ferreira, W. van Mechelen, M. H. Prins, J. W. Twisk, and C. D. Stehouwer
Lifetime Vigorous But Not Light-To-Moderate Habitual Physical Activity Impacts Favorably on Carotid Stiffness in Young Adults: The Amsterdam Growth and Health Longitudinal Study
Hypertension, January 1, 2010; 55(1): 33 - 39.
[Abstract] [Full Text] [PDF]

Home page
 BMJHome page
L. Byberg, H. Melhus, R. Gedeborg, J. Sundstrom, A. Ahlbom, B. Zethelius, L. G Berglund, A. Wolk, and K. Michaelsson
Total mortality after changes in leisure time physical activity in 50 year old men: 35 year follow-up of population based cohort
BMJ, March 5, 2009; 338(mar05_2): b688 - b688.
[Abstract] [Full Text] [PDF]

Home page
 JCOHome page
A. N. Freedman, M. L. Slattery, R. Ballard-Barbash, G. Willis, B. J. Cann, D. Pee, M. H. Gail, and R. M. Pfeiffer
Colorectal Cancer Risk Prediction Tool for White Men and Women Without Known Susceptibility
J. Clin. Oncol., February 10, 2009; 27(5): 686 - 693.
[Abstract] [Full Text] [PDF]

Home page
 J Aging HealthHome page
C. F. Mendes de Leon, K. A. Cagney, J. L. Bienias, L. L. Barnes, K. A. Skarupski, P. A. Scherr, and D. A. Evans
Neighborhood Social Cohesion and Disorder in Relation to Walking in Community-Dwelling Older Adults: A Multilevel Analysis
J Aging Health, February 1, 2009; 21(1): 155 - 171.
[Abstract] [PDF]

Home page
 Br. J. Sports. Med.Home page
B E Ainsworth
How do I measure physical activity in my patients? Questionnaires and objective methods
Br. J. Sports Med., January 1, 2009; 43(1): 6 - 9.
[Abstract] [Full Text] [PDF]

Home page
 Cancer Epidemiol. Biomarkers Prev.Home page
Y. Bao and D. S. Michaud
Physical Activity and Pancreatic Cancer Risk: A Systematic Review
Cancer Epidemiol. Biomarkers Prev., October 1, 2008; 17(10): 2671 - 2682.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
166/7/832    most recent
kwm148v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (17)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Ferrari, P.
Right arrow Articles by Matthews, C. E.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ferrari, P.
Right arrow Articles by Matthews, C. E.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?