American Journal of Epidemiology

American Journal of Epidemiology Advance Access originally published online on July 5, 2007
American Journal of Epidemiology 2007 166(6):646-655; doi:10.1093/aje/kwm165

This Article Abstract FREE Full Text (PDF) All Versions of this Article: 166/6/646    most recent kwm165v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Related articles in Am. J. Epidemiol. Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Search for citing articles in: ISI Web of Science (27) Request Permissions Disclaimer Google Scholar Articles by Fewell, Z. Articles by Sterne, J. A. C. Search for Related Content PubMed PubMed Citation Articles by Fewell, Z. Articles by Sterne, J. A. C. Social Bookmarking          What's this?

American Journal of Epidemiology © The Author 2007. Published by the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

PRACTICE OF EPIDEMIOLOGY

The Impact of Residual and Unmeasured Confounding in Epidemiologic Studies: A Simulation Study

Zoe Fewell, George Davey Smith and Jonathan A. C. Sterne

From the Department of Social Medicine, University of Bristol, Bristol, United Kingdom

Correspondence to Prof. Jonathan A. C. Sterne, Department of Social Medicine, University of Bristol, Canynge Hall, Whiteladies Road, Bristol BS8 2PR, United Kingdom (e-mail: Jonathan.Sterne{at}bristol.ac.uk).

Received for publication April 11, 2005. Accepted for publication November 11, 2005.

	ABSTRACT

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
Measurement error in explanatory variables and unmeasured confounders can cause considerable problems in epidemiologic studies. It is well recognized that under certain conditions, nondifferential measurement error in the exposure variable produces bias towards the null. Measurement error in confounders will lead to residual confounding, but this is not a straightforward issue, and it is not clear in which direction the bias will point. Unmeasured confounders further complicate matters. There has been discussion about the amount of bias in exposure effect estimates that can plausibly occur due to residual or unmeasured confounding. In this paper, the authors use simulation studies and logistic regression analyses to investigate the size of the apparent exposure-outcome association that can occur when in truth the exposure has no causal effect on the outcome. The authors consider two cases with a normally distributed exposure and either two or four normally distributed confounders. When the confounders are uncorrelated, bias in the exposure effect estimate increases as the amount of residual and unmeasured confounding increases. Patterns are more complex for correlated confounders. With plausible assumptions, effect sizes of the magnitude frequently reported in observational epidemiologic studies can be generated by residual and/or unmeasured confounding alone.

bias (epidemiology); computer simulation; confounding factors (epidemiology); logistic models

---

Abbreviations: ICC, intraclass correlation coefficient

	INTRODUCTION

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
The aim of epidemiologic analysis is often to estimate the causal effect of an exposure on an outcome of interest. The problems involved in using observational studies for causal inference are well known (1, 2). Selection bias, recall bias, loss to follow-up, and reverse causation are some of the problems that can lead to biased associations between exposure and outcome in observational studies. Confounding can be caused by variables that are associated with both outcome and exposure and are not on the causal pathway between exposure and outcome. Controlling for variables with these properties may remove bias, but the investigators must both perfectly measure all of them and perfectly characterize their association with the exposure of interest. There are also variables for which control does not remove bias even though they have all three properties of confounders (3). While it is recognized that, under certain conditions, nondifferential measurement error in the exposure leads to a bias towards the null (4), the effects of measurement error in confounders, which leads to residual confounding, are not well understood.

Consider, for example, the effect of antioxidant vitamin intake on the risks of cancer, cardiovascular disease, and mortality. Observational studies have shown protective effects against these outcomes (5–7), while in contrast, randomized trials have shown no effect (8, 9). It has been suggested that the disparity in results is likely to be due to confounding by behavioral and social factors acting across the life course (10, 11). For example, factors related to childhood social class may be important confounders of the association between antioxidant vitamin intake and disease outcome. To fully capture the life-course effect of such confounders, all related factors must be measured perfectly. Failure to do so, either because of unmeasured confounders or because of residual confounding, will result in biased exposure effect estimates.

There are, of course, other possible explanations for this disparity in results. Observational studies estimate the effect of dietary exposure on disease outcomes, whereas randomized trials change exposure in the intervention group. This change is often made through supplementation, and can result in exposure to the nutrient of interest much higher than would ever be seen in an observational study. In some situations, however, the exposure in the observational studies is precisely the same as the treatment in the randomized controlled trials. For example, two widely cited 1993 studies demonstrated substantially lower coronary heart disease risks among people using vitamin E supplements that were apparently robust to confounding (6, 7). Furthermore, there was no trend towards increasing protection by vitamin E supplements when they were taken for more than 2 years. The exposure in these observational studies was precisely the same as that tested in randomized controlled trials of vitamin E supplementation that have run for up to 6 years. In these randomized controlled trials, there is robust evidence of no material effect on coronary heart disease risk (12).

Some authors debate whether residual confounding can cause large exposure-outcome effect estimates. Morabia (13) asserts that strong associations are unlikely to be completely attributable to confounding, because strong confounders are likely to be detected in the study population or recognized in the literature as strong confounders and therefore measured and controlled for in the analysis. More recently, Khaw et al. (14) advanced similar arguments with respect to whether residual confounding could explain observed associations between plasma ascorbic acid and mortality.

It is well documented that measurement error in confounders reduces our ability to control for confounding in the analysis. For example, Greenland (15) and Brenner (16) have illustrated the effects of misclassification of dichotomous and polytomous confounders, respectively. Savitz and Barón (17) have proposed a method for correcting for confounder misclassification with dichotomous exposure variables, disease variables, and confounding variables. Phillips and Davey Smith (18) considered measurement error in a continuous exposure and a confounding variable. Marshall and Hastrup (19) considered three cases, with either continuous or dichotomous outcomes, exposures, and confounders, in which an exposure and a strong confounder are affected by uncorrelated measurement errors. Kipnis et al. (20) investigated the impact of measurement error in the exposure and a confounder on the exposure regression coefficient in linear regression models. Marshall et al. (21) examined two cases in which the exposure and confounder are continuous, the outcome variable is either continuous or dichotomous, and errors in variables are correlated.

All of the above articles concentrated on data in which there is only one confounder, in which case the assertions made by Morabia (13) may generally (but not always) apply. Investigators have also examined the effects of residual confounding on exposure effect estimates in multivariable models. For example, measurement error in exposure and confounders in failure time models was considered by Prentice (22). Armstrong et al. (23) proposed a method of correcting for measurement error in normally distributed exposures and confounders and provided general formulae for the effect of measurement error on regression coefficients under their assumption of a normal discriminant analysis model. Spiegelman et al. (24) described a general method of correcting for measurement error in continuous or categorical exposures and confounders.

Throughout this paper, we use the term residual confounding to refer to confounding due to measurement error in a confounder included in the model and unmeasured confounding to refer to confounding due to omission of a confounder from the model. We examine the effects of residual and unmeasured confounding on the estimated exposure effect when there are several confounders, each of which may be affected by measurement error or omitted from the analysis.

	MATERIALS AND METHODS

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
We denote the exposure variable as E and the confounders as X_i, i = 1, ..., n. Figure 1 shows a common situation in epidemiologic studies. The dashed arrows indicate possible correlations between exposure and confounders. Often several factors are measured as surrogates for a causal factor, or a causal factor is measured directly but with error. The measured factors are denoted by Z_E and Z_i, i = 1, ..., p, and may also be correlated. We use the measured factors to estimate the effect of the causal factors on the dichotomous outcome variable Y. We concentrate on the simple case where n = p and each causal factor is represented by one measured factor (figure 2). The causal factors, and therefore the measured factors, can be correlated, as indicated by the dashed arrows.

View larger version (8K): [in this window] [in a new window] [Download PowerPoint slide]   FIGURE 1. General relations between causal factors, measured factors, and outcome in an epidemiologic study. In the general case, the number of causal factors may differ from the number of measured factors. The measured factors are less-than-perfect substitutes for the causal factors.

 

View larger version (5K): [in this window] [in a new window] [Download PowerPoint slide]   FIGURE 2. Simplified relations between causal factors, measured factors, and outcome in an epidemiologic study, as assumed in the current analysis. The measured factors are less-than-perfect substitutes for the causal factors.

 
We assume that the casual factors and the outcome are related by the logistic model

(1)

Here, p is the probability of the outcome. The exposure and confounders are all normally distributed with mean 0 and variance 1. The true exposure-outcome log odds ratio, ß_E, is 0, while the true confounder-outcome log odds ratios, ß₁, ..., ß_n, are ln 2. The value of is set to ln 0.1 throughout. Other than changing the number of outcome events in the data set, the choice of is not important.

Measurement error can be quantified using the intraclass correlation coefficient (ICC), defined as

where is the variance of the true measurements and is the error variance. An ICC of 1 implies that = 0 and that there is no measurement error. If = , the ICC will equal 0.5. An ICC of 0 implies that = 0 and that the variation in the variable is entirely due to error. In epidemiologic studies with paired measurements, the usual method of estimating the ICC is to calculate the Pearson correlation with each pair entered twice, once in reverse order. Alternatively, the ICC can be estimated using one-way analysis of variance or by using a simple random-effects model (25). We assume that the exposure, E, is measured without error, and we introduce error into the confounders, X₁, ..., X_n, to create variables Z₁, ..., Z_n.

(2)

where corr(_i, _j) = 0 if i j (i.e., the measurement errors are independent of each other). For simplicity, we also assume that _i N(0, ) and do not consider different distributions for the errors. The error variance, , is either 1 or 1/3, corresponding to ICCs of 0.5 and 0.75, respectively. These are plausible values for ICCs that can occur in epidemiologic studies. For example, Satia-Abouta et al. (26) showed test-retest ICCs of 0.74 for iron and 0.76 for chromium for mean supplement intake over a 10-year period as recorded by self-administered questionnaire. Schroder et al. (27) showed an ICC of 0.49 for percentage of dietary energy intake made up by carbohydrate when measured by 72-hour recall and an ICC of 0.52 for percentage of energy intake made up by fat when using a food frequency questionnaire. Among women completing the Lifetime Drinking History questionnaire, Friesema et al. (28) showed test-retest ICCs of 0.75 for frequency of alcohol drinking in adulthood and 0.50 for quantity of drinking at age 61 years or more. There are many other examples of ICCs of these sizes in the literature (29–34).

The first stage in generating the simulated data sets was to draw the exposure E, the confounders X_i, and the error variables _i from a multivariate normal distribution, where the correlation between each pair of variables was specified and 500,000 observations per data set were generated. The mismeasured confounders were then created according to equation 2. To generate the dichotomous outcome variable, we first generated a uniformly distributed variable between 0 and 1 for each observation in the data set. We then calculated the probability of the outcome, p, for each observation using equation 1 and the values of the parameters specified previously. From the properties of the uniform distribution, the probability of the uniform variable being less than p equals p. We therefore defined the outcome to be 1 if the uniform variable was less than p and 0 otherwise. This simulates a Bernoulli trial for each observation in the data set and creates an outcome variable that has the correct probability, p, of being equal to 1.

The correlations between the exposure E and the confounders were assumed to be 0.1, 0.3, or 0.5, while the correlation between confounders was either 0 or 0.5. Such correlations are not uncommon in epidemiologic studies. For example, in a study of 3,524 children from the Child and Adolescent Trial for Cardiovascular Health, Osganian et al. (35) reported a correlation between serum folic acid level and vitamin B₆ of 0.48 and a correlation between serum homocysteine level and body mass index of 0.09. They also reported that there was no correlation between serum homocysteine level and diastolic blood pressure or serum lipid levels. Variables from the British Women's Heart and Health Study also showed correlations of sizes similar to those considered here. For example, the correlation between total serum protein concentration and diastolic blood pressure was 0.10, that between weight at age 21 years and present weight was 0.30, and that between serum albumin concentration and mean cellular hemoglobin concentration was 0.49 (Debbie Lawlor, University of Bristol, unpublished data). No generality is lost by considering only positive correlations between an exposure and confounders; the corresponding results for negative correlation can be obtained by inverting odds ratios. Tables of results for correlations between confounders of 0.1, 0.2, 0.3, or 0.4 are available from the authors.

Once the data sets had been simulated, logistic regression was used to estimate the crude exposure-outcome odds ratio and the odds ratios adjusted for confounders measured with or without error. Each data set used in the analyses was simulated 50 times, using a different random number seed on each occasion. Odds ratios presented in this paper are the geometric mean values of the odds ratios from these 50 simulations. Fifty simulations each of 500,000 observations produced small 95 percent confidence intervals around the geometric means, with widths of no more than 0.011. We used Stata 8.2 (Stata Corporation, College Station, Texas) for all analyses, employing the corr2data command with the seed option to generate the normally distributed variables. We considered two situations in which there were either two confounders or four confounders.

	RESULTS

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
Two confounders
We start by considering a situation where there are two confounding variables, X₁ and X₂.

Table 1 displays estimated odds ratios for the association between E and the binary outcome, based on simulations in which the confounders are uncorrelated. For each combination of correlations between E and X₁ and E and X₂, the crude odds ratio, the odds ratio adjusted for Z₁ alone, and the odds ratio adjusted for both Z₁ and Z₂ are shown. Note that when the ICCs for both Z₁ and Z₂ equal 1, then Z₁ = X₁ and Z₂ = X₂, so the adjusted odds ratio for E is equal to 1 (no residual confounding). In all other situations, unmeasured and/or residual confounding biases the estimated odds ratio away from 1. The crude odds ratios increase with increasing correlations of E with X₁ and X₂ and are symmetric with respect to these correlations. When the correlations of E with X₁ and X₂ are each 0.5, the maximum crude odds ratio is 1.93.

View this table: [in this window] [in a new window]   TABLE 1. Estimated odds ratios in a situation with two confounders for a single standard-deviation increase in E when adjusting for Z1 alone or for Z1 and Z2, according to the correlation between confounders and exposure and the amount of measurement error in the confounders (correlation between X1 and X2 = 0)*

 
There are nine sets of nine odds ratios adjusted for both Z₁ and Z₂; these correspond to the residual confounding caused by imperfect measurement of the two confounders. When the correlation between E and X₁ is equal to the correlation between E and X₂, the odds ratios adjusted for Z₁ and Z₂ are symmetric with respect to the measurement error in Z₁ and Z₂. Where the correlation between E and X₁ is not equal to the correlation between E and X₂, the odds ratios adjusted for Z₁ and Z₂ are asymmetric with respect to the measurement error. For example, when the correlation of X₁ with E is 0.5 and the correlation of X₂ with E is 0.1, the odds ratio adjusted for both Z₁ and Z₂ is 1.05 when Z₁ is measured with ICC equal to 1 and Z₂ is measured with ICC equal to 0.5. When the ICC for Z₁ is 0.5 and the ICC for Z₂ is 1, the odds ratio adjusted for both Z₁ and Z₂ is 1.22.

In general, the odds ratios adjusted for both Z₁ and Z₂ are larger when there is more measurement error (a smaller ICC) and when the correlations between E and the confounders are higher. In addition, the odds ratios adjusted for Z₁ alone are larger than the odds ratios adjusted for both Z₁ and Z₂ because of unmeasured confounding.

Having considered the situation in which the two confounders are uncorrelated, we now examine the implications of a correlation between the confounders of 0.5. Table 2 shows the crude odds ratio, the odds ratio adjusted for Z₁ alone, and the odds ratio adjusted for both Z₁ and Z₂ for each combination of correlations of E with X₁ and X₂. Because of the correlation between the confounders, the odds ratios displayed in table 2 are generally smaller than those in table 1. When the correlations of E with X₁ and X₂ are 0.5, the maximum crude odds ratio is 1.85.

View this table: [in this window] [in a new window]   TABLE 2. Estimated odds ratios in a situation with two confounders for a single standard-deviation increase in E when adjusting for Z1 alone or for Z1 and Z2, according to the correlation between confounders and exposure and the amount of measurement error in the confounders (correlation between X1 and X2 = 0.5)*

 
In general, as the correlation of E with X₁ and X₂ increases, the odds ratio adjusted for Z₁ alone or both Z₁ and Z₂ increases. There are exceptions to this general rule. For example, when the correlation between E and X₁ is 0.1, the correlation between E and X₂ is 0.5, and Z₁ is measured without error (ICC = 1), the odds ratio adjusted for Z₁ only is 1.36 (crude odds ratio = 1.43). This is greater than the corresponding odds ratio of 1.25, when the correlation between E and X₁ is 0.5, even though the corresponding crude odds ratio is 1.85. This effect occurs because X₁ is perfectly measured (ICC = 1) and is also correlated with X₂. The increased confounding present when the correlation between E and X₁ is 0.5 is offset by the correspondingly improved indirect control for the unmeasured confounder (X₂).

Generally, more measurement error in Z₁ or Z₂ (a smaller ICC) results in larger estimated odds ratios (more residual confounding). Again, there are exceptions to this general rule. For example, when the correlation between E and X₁ is 0.1, the correlation between E and X₂ is 0.5, Z₁ is measured without error (ICC = 1), and Z₂ is measured with ICC = 0.5, the odds ratio adjusted for both Z₁ and Z₂ is 1.22. When the ICC for Z₁ is 0.5 (measurement error increases), the odds ratio adjusted for Z₁ and Z₂ decreases to 1.20. With these correlations between confounders and between confounders and exposure, the partial correlation between E and X₁, conditional on X₂, is negative. In this situation, the effect of increasing X₁ while keeping X₂ fixed is a reduction in the estimated exposure-outcome odds ratio. It follows that as measurement error in Z₁ increases, the estimated odds ratio decreases.

Although, in general, unmeasured confounding results in larger estimated odds ratios, there are examples where the odds ratio is smaller with greater unmeasured confounding. When the correlation between E and X₁ is 0.5, the correlation between E and X₂ is 0.1, and ICC = 0.5 for both confounders, the odds ratio adjusted for Z₁ and Z₂ is 1.20. When adjusting only for Z₁, the odds ratio decreases to 1.17. Again, this combination of correlations leads to a negative partial correlation between E and X₂, conditional on X₁. Omitting X₂ from the analysis therefore decreases the estimated exposure-outcome odds ratio.

Intuitively, we would expect that either unmeasured confounding or residual confounding would lead to imperfect control and hence to adjusted odds ratios that were intermediate between the crude odds ratio and the correct value of 1.0. However, in table 2, estimated odds ratios less than 1 occur when the correlation of E with one of the confounders is 0.1 and the correlation of E with the other confounder is either 0.3 or 0.5. This effect occurs when one confounder is weak (i.e., has a correlation of 0.1 with exposure) and the other is both stronger (a correlation of 0.3 or 0.5 with exposure) and measured without error. Again, this is due to a negative partial correlation between E and the weaker confounder.

Four confounders
We now consider the results of simulations including four confounders. To reduce the number of data sets to be simulated, all correlations between pairs of confounders were assumed to be equal and were again either 0 or 0.5. We estimated the exposure-outcome association while controlling for either one or two confounders in the analyses.

Table 3 displays estimated odds ratios for the association between exposure and the binary outcome, based on simulations in which the confounders are uncorrelated. For each combination of correlations between E and X₁ and E and X₂, the crude odds ratio, the odds ratio adjusted for Z₁ only, and the odds ratio adjusted for Z₁ and Z₂ are shown. To ensure that the correlation matrix was positive definite, we did not simulate data sets where the correlation between all confounders and exposure was 0.5. Table 3 therefore displays results for situations where the correlation between E and X₃ is 0.5 and the correlation between E and X₄ is 0.3, as these were the highest correlations available that held for all correlations of E with X₁ and X₂.

View this table: [in this window] [in a new window]   TABLE 3. Estimated odds ratios in a situation with four confounders for a single standard-deviation increase in E when adjusting for Z1 alone or for Z1 and Z2, according to the correlation between confounders and exposure and the amount of measurement error in the confounders (correlation between pairs of confounders = 0)*

 
Because there is always unmeasured confounding by X₃ and X₄, the estimated odds ratios in table 3 never equal the correct value of 1. In general, the odds ratios adjusted for Z₁ alone or for both Z₁ and Z₂ are larger when the correlations of E with X₁ and X₂ are larger. More measurement error (a smaller ICC) results in larger estimated odds ratios, and unmeasured confounding increases the estimated odds ratios. The estimated odds ratios are larger than those shown in either table 1 or table 2 because of the larger amount of unmeasured confounding, and we see crude odds ratios as large as 3.27. Residual confounding is now relatively unimportant compared with unmeasured confounding.

Table 4 shows results from simulations in which the correlation between the confounders is 0.5. To enable comparison between table 3 and table 4, the correlations between E and X₃ and E and X₄ were again set to 0.5 and 0.3, respectively.

View this table: [in this window] [in a new window]   TABLE 4. Estimated odds ratios in a situation with four confounders for a single standard-deviation increase in E when adjusting for Z1 alone or for Z1 and Z2, according to the correlation between confounders and exposure and the amount of measurement error in the confounders (correlation between pairs of confounders = 0.5)*

 
Because of the correlation between the confounders, the odds ratios displayed in table 4 are smaller than those in table 3. However, they are larger than those shown in tables 1 and 2 because of the larger amount of unmeasured confounding. Although the relations of the degree of bias to residual and unmeasured confounding observed in table 3 still hold in general, there are exceptions. There are instances of increasing measurement error (decreasing ICC) or increasing correlation between exposure and confounders leading to smaller estimated odds ratios. There are also examples of unmeasured confounding leading to decreases in the estimated odds ratio. As was discussed in detail in the context of table 2, the correlations between the underlying confounders X₁ to X₄ lead to complex relations of the adjusted odds ratios with the strength of confounding (correlation with E) and with measurement error in the confounders.

Figure 3 displays the effect of controlling for all four confounders, where the confounders are measured with varying amounts of error. For simplicity, all confounders are assumed to have the same ICC and the correlations between all pairs of confounders are assumed to be equal, unless the confounder-exposure correlation is 0.5. In this case, the correlation between E and X₄ is 0.3 and the correlation between all other confounders and exposure is 0.5. The residual confounding of the exposure-outcome odds ratio increases with the measurement error. The adjusted odds ratio increases as the correlation of each confounder with exposure increases and as the correlations between pairs of confounders decrease. The largest adjusted odds ratio of 2.81 is observed when the ICC equals 0.5, the confounders are uncorrelated, and the confounder-exposure correlation is 0.5. Figure 3 shows that exposure effects may be estimated with substantial bias because of residual confounding alone.

View larger version (14K): [in this window] [in a new window] [Download PowerPoint slide]   FIGURE 3. Relation between the exposure-outcome odds ratio (OR) adjusted for all four confounders and confounder measurement error, correlation between confounders, and confounder-exposure correlation. Measurement error and confounder-exposure correlations are assumed to be equal for all confounders. For a confounder-exposure correlation of 0.5, the correlation between E and X4 is 0.3. The ORs on the y-axis are plotted on a log scale. ICC, intraclass correlation coefficient.

 
The effect of different numbers of unmeasured confounders on the estimated exposure-outcome odds ratio is displayed in figure 4. For simplicity, all confounders are assumed to be measured without error, and the correlations between each confounder and exposure are assumed to be equal, except when the confounders have a correlation of 0.5, in which case the correlation between E and X₄ is 0.3 while all other confounders have a correlation of 0.5 with exposure. The estimated odds ratio increases as the number of confounders controlled for decreases and as the correlation between confounders and exposure increases. Bias due to unmeasured confounding is worse when the confounders are uncorrelated. When the exposure-confounder correlation is 0.5, there is serious bias in the estimated exposure-outcome odds ratio of 2.22, even when three confounders are controlled for.

View larger version (11K): [in this window] [in a new window] [Download PowerPoint slide]   FIGURE 4. Relation between the exposure-outcome odds ratio (OR) and the number of confounders controlled for, confounder-exposure correlation, and correlation between pairs of confounders. All confounders are assumed to be measured without error, and the confounder-exposure correlations are assumed to be equal for all confounders. The ORs on the y-axis are plotted on a log scale.

 

	DISCUSSION

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
The validity of an epidemiologic study may be threatened by both residual and unmeasured confounding. With plausible assumptions about residual and unmeasured confounding, effect sizes of the magnitude frequently reported in observational epidemiologic studies can be generated. This study has shown that if the confounders are uncorrelated, bias in the estimated exposure-outcome odds ratio increases as error in the measured confounders increases, as the number of unmeasured confounders increases, and as the correlation of the confounders with exposure increases. If the confounders are correlated, bias in the estimated odds ratio can decrease as measurement error increases, as the correlation between confounders and exposure increases, and as unmeasured confounding increases. Unmeasured confounding is a more serious problem when the confounders are uncorrelated, and it can result in substantial bias in the estimated exposure-outcome odds ratio, even when only one confounder is omitted from the analysis.

Usually, when one is analyzing the results of an epidemiologic study, the true model is not known. We do not reliably know which variables are confounders of the association of interest, the form in which they should enter the model, or the time scale over which they act. It has been suggested that confounders can be identified by evaluating the change in the exposure-outcome estimate (36)—for example, if the estimate adjusted for a variable differs by more than 15 percent from the estimate obtained without adjusting for that variable, the variable should be considered a confounder. Strict adherence to such a rule could lead to true confounders' being disregarded. Consider table 1, for example. If the correlations between both E and X₁ and E and X₂ were 0.1, the odds ratio adjusted for each of them separately would only differ from the crude odds ratio by 2.7–5.6 percent, depending on measurement error. This would lead us to believe that X₁ and X₂ were not confounders and that the crude odds ratio was the true exposure effect estimate. Rules such as the change-in-estimate criterion should be applied carefully or not at all (37).

We have considered only the case in which the errors in confounders are uncorrelated. Returning to the motivating example of the effect of antioxidant vitamins on cardiovascular outcomes, this assumption may not be realistic. If quantities of nutrients are derived from a questionnaire containing questions about the frequency and quantity of consumption of certain food types, errors in variables may well be correlated. Errors in reporting on a type of food that contains two nutrients will result in the errors in the quantities of those two nutrients being correlated.

Here, we have used simulation studies to illustrate our results. While analytic results are always desirable, they are not always possible to obtain. As Gustafson stated, "Unfortunately, closed-form expressions for the bias induced by measurement error in logistic regression do not exist" (38, p. 24). Our aim here was to examine the effects of unmeasured and residual confounding given parameter combinations representing situations commonly seen in epidemiologic research.

Care should be taken in making generalizations from the results of a simulation study (39). Here, we assumed that both exposure and confounders were normally distributed with mean 0 and variance 1. Continuous variables in epidemiologic studies may not have a normal distribution, but logarithm or square-root transformations can improve normality (although this can cause added difficulty in interpreting the results of a regression analysis). Our results do not depend on the choice of mean and can be interpreted, without loss of generality, as the effect of a single standard-deviation increase in the exposure and confounders. Often, in epidemiologic studies, some or all of the confounders and/or the exposure are categorical. The results presented here will not generally apply in this situation. Misclassification of categorical (or binary) variables is a more complex problem, since errors will be correlated with the true values (40). Extensions of this work that addressed the issue of categorical exposures and confounders would be desirable.

Our aim in this study was to show the magnitude of bias that can result from residual and unmeasured confounding. Therefore, we assumed throughout that exposure is measured without error, which may not be a realistic assumption. As Savitz and Barón (17) noted, the effect of measurement error in all variables should be considered. The investigator should consider whether the effects of measurement error in confounders and exposure act in the same direction or in opposite directions. With large numbers of confounders, this is a complex problem.

This study highlights the need to perform sensitivity analyses to assess whether unmeasured and residual confounding are likely problems. We have shown that unmeasured confounders have a cumulative effect on the bias of exposure effect estimates. The possibility of the presence of several unmeasured confounders should be taken into account when performing sensitivity analyses. It may not be enough to state that a single unmeasured confounder would need an implausibly large odds ratio to remove the observed effect. Several unmeasured confounders with small or moderate effects may be able to produce the same effects. Sensitivity analysis methods for assessing the possible effects of selection bias, misclassification of covariates, and unmeasured confounding have been proposed and illustrated by Greenland (41) and Lash and Fink (42).

If information on confounders is available, it should be used in the estimation of effects. For example, in a study of the relation between plasma ascorbic acid level and mortality, Khaw et al. (5) noted that while information on the social class and physical activity of the participants was recorded, it was not used in the analysis. If these variables were indeed confounders of the relation between ascorbic acid and mortality, even a moderate effect of each would have resulted in sizable residual confounding in the reported estimates, as Lawlor et al. (11) showed. Confounders may be omitted from the analysis because of missing data leading to loss of information. In these situations, methods of dealing with missing data (e.g., multiple imputation (43, 44)) can be used to avoid bias due to unmeasured confounding.

The effect of measurement error on exposure effect estimates should be explored, either by adjusting the estimates based on knowledge of the likely measurement error or by performing sensitivity analyses. Of course, the ideal circumstance is that the variables are measured without error, but this is unlikely to occur in reality. While efforts should be made to minimize measurement error, the measurement error that has occurred should be quantified and used in the final effect estimate.

	ACKNOWLEDGMENTS

 
This work was supported by a research studentship from the Medical Research Council.

Conflict of interest: none declared.

	NOTES

 
Editor's note: An invited commentary on this article appears on page 656, and the authors' response appears on page 659.

	References

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 

1. Beaglehole R, Bonita R, Kjellström T. Causation in epidemiology. In: Basic epidemiology. (1993) Geneva, Switzerland: World Health Organization. 71–81.
2. Kelsey JL, Petitti DB, King AC. Key methodologic concepts and issues. In: Applied epidemiology—Brownson RC, Petitti DB, eds. (1998) New York, NY: Oxford University Press. 35–69.
3. Greenland S, Robins JM. Identifiability, exchangeability, and epidemiologic confounding. Int J Epidemiol (1986) 15:413–19.[Abstract/Free Full Text]
4. Jurek AM, Greenland S, Maldonado G, et al. Proper interpretation of non-differential misclassification effects: expectations vs observations. Int J Epidemiol (2005) 34:680–7.[Abstract/Free Full Text]
5. Khaw KT, Bingham S, Welch A, et al. Relation between plasma ascorbic acid and mortality in men and women in EPIC-Norfolk prospective study: a prospective population study. Lancet (2001) 357:657–63.[CrossRef][Web of Science][Medline]
6. Rimm EB, Stampfer MJ, Ascherio A, et al. Vitamin E consumption and the risk of coronary heart disease in men. N Engl J Med (1993) 328:1450–6.[Abstract/Free Full Text]
7. Stampfer MJ, Hennekens CH, Manson JE, et al. Vitamin E consumption and the risk of coronary disease in women. N Engl J Med (1993) 328:1444–9.[Abstract/Free Full Text]
8. Heart Protection Study Collaborative. Group. MRC/BHF Heart Protection Study of antioxidant vitamin supplementation in 2053 6 high-risk individuals: a randomised placebo-controlled trial. Lancet (2002) 360:23–33.[CrossRef][Web of Science][Medline]
9. Collaborative Group of the Primary Prevention Project. Low-dose aspirin and vitamin E in people at cardiovascular risk: a randomised trial in general practice. Lancet (2001) 357:89–95.[CrossRef][Web of Science][Medline]
10. Davey Smith G. Reflections on the limitations to epidemiology. J Clin Epidemiol (2001) 54:325–31.[CrossRef][Web of Science][Medline]
11. Lawlor DA, Davey Smith G, Bruckdorfer KR, et al. Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? Lancet (2004) 363:1724–7.[CrossRef][Web of Science][Medline]
12. Eidelman RS, Hollar D, Hebert PR, et al. Randomized trials of vitamin E in the treatment and prevention of cardiovascular disease. Arch Intern Med (2004) 164:1552–6.[Abstract/Free Full Text]
13. Poppers Morabia A. Kaposi's sarcoma, and HIV infection: empirical example of a strong confounding effect? Prev Med (1995) 24:90–5.[CrossRef][Web of Science][Medline]
14. Khaw KT, Day N, Bingham S, et al. Observational versus randomised trial evidence. Lancet (2004) 364:753–4.[Web of Science][Medline]
15. Greenland S. The effect of misclassification in the presence of covariates. Am J Epidemiol (1980) 112:564–9.[Abstract/Free Full Text]
16. Brenner H. Bias due to nondifferential misclassification of polytomous confounders. J Clin Epidemiol (1993) 46:57–63.[CrossRef][Web of Science][Medline]
17. Savitz DA, Barón AE. Estimating and correcting for confounder misclassification. Am J Epidemiol (1989) 129:1062–71.[Abstract/Free Full Text]
18. Phillips AN, Davey Smith G. Bias in relative odds estimation owing to imprecise measurement of correlated exposures. Stat Med (1992) 11:953–61.[Web of Science][Medline]
19. Marshall JR, Hastrup JL. Mismeasurement and the resonance of strong confounders: uncorrelated errors. Am J Epidemiol (1996) 143:1069–78.[Abstract/Free Full Text]
20. Kipnis V, Freedman LS, Brown CC, et al. Effect of measurement error on energy-adjustment models in nutritional epidemiology. Am J Epidemiol (1997) 146:842–55.[Abstract/Free Full Text]
21. Marshall JR, Hastrup JL, Ross JS. Mismeasurement and the resonance of strong confounders: correlated errors. Am J Epidemiol (1999) 150:88–96.[Abstract/Free Full Text]
22. Prentice RL. Covariate measurement errors and parameter-estimation in a failure time regression-model. Biometrika (1982) 69:331–42.[Abstract/Free Full Text]
23. Armstrong BG, Whittemore AS, Howe GR. Analysis of case-control data with covariate measurement error—application to diet and colon cancer. Stat Med (1989) 8:1151–63.[Web of Science][Medline]
24. Spiegelman D, Rosner B, Logan R. Estimation and inference for logistic regression with covariate misclassification and measurement error in main study/validation study designs. J Am Stat Assoc (2000) 95:51–61.[CrossRef][Web of Science]
25. Kirkwood BR, Sterne JAC. Measurement error: assessment and implications. In: Essentials of medical statistics. (2003) 2nd ed. London, United Kingdom: Blackwell Science Ltd. 429–46.
26. Satia-Abouta J, Patterson RE, King IB, et al. Reliability and validity of self-report of vitamin and mineral supplement use in the Vitamins and Lifestyle Study. Am J Epidemiol (2003) 157:944–54.[Abstract/Free Full Text]
27. Schroder H, Covas MI, Marrugat J, et al. Use of a three-day estimated food record, a 72-hour recall and a food-frequency questionnaire for dietary assessment in a Mediterranean Spanish population. Clin Nutr (2001) 20:429–37.[CrossRef][Web of Science][Medline]
28. Friesema IH, Veenstra MY, Zwietering PJ, et al. Measurement of lifetime alcohol intake: utility of a self-administered questionnaire. Am J Epidemiol (2004) 159:809–17.[Abstract/Free Full Text]
29. Chinn S, Schouten JP. Reproducibility of non-specific bronchial challenge in adults: implications for design, analysis and interpretation of clinical and epidemiological studies. Thorax (2005) 60:395–400.[Abstract/Free Full Text]
30. Freeman R, Chase KP, Risk MR. Quantitative sensory testing cannot differentiate simulated sensory loss from sensory neuropathy. Neurology (2003) 60:465–70.[Abstract/Free Full Text]
31. Gallagher KM, Jara M, Demaria A, et al. The reliability of passively collected AIDS surveillance data in Massachusetts. Ann Epidemiol (2003) 13:100–4.[CrossRef][Web of Science][Medline]
32. Salerno DF, Franzblau A, Armstrong TJ, et al. Test-retest reliability of the upper extremity questionnaire among keyboard operators. Am J Ind Med (2001) 40:655–66.[CrossRef][Web of Science][Medline]
33. Yokoo EM, Valente JG, Sichieri R, et al. Validation and calibration of mercury intake through self-referred fish consumption in riverine populations in Pantanal Mato-grossense, Brazil. Environ Res (2001) 86:88–93.[Medline]
34. Xu LZ, Porteous JE, Phillips MR, et al. Development and validation of a calcium intake questionnaire for postmenopausal women in China. Ann Epidemiol (2000) 10:169–75.[CrossRef][Web of Science][Medline]
35. Osganian SK, Stampfer MJ, Spiegelman D, et al. Distribution of and factors associated with serum homocysteine levels in children. JAMA (1999) 281:1189–96.[Abstract/Free Full Text]
36. Sonis J. A closer look at confounding. Fam Med (1998) 30:584–8.[Medline]
37. Hernan MA, Hernandez-Diaz S, Werler MM, et al. Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology. Am J Epidemiol (2002) 155:176–84.[Abstract/Free Full Text]
38. Gustafson P. Measurement error and misclassification in statistics and epidemiology. (2004) London, United Kingdom: Chapman & Hall Ltd.
39. Maldonado G, Greenland S. The importance of critically interpreting simulation studies. Epidemiology (1997) 8:453–6.[Web of Science][Medline]
40. White I, Frost C, Tokunaga S. Correcting for measurement error in binary and continuous variables using replicates. Stat Med (2001) 20:3441–57.[CrossRef][Web of Science][Medline]
41. Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol (1996) 25:1107–16.[Abstract/Free Full Text]
42. Lash TL, Fink AK. Semi-automated sensitivity analysis to assess systematic errors in observational data. Epidemiology (2003) 14:451–8.[Web of Science][Medline]
43. Rubin DB. Multiple imputation for nonresponse in surveys. (1987) New York, NY: John Wiley and Sons, Inc.
44. Schafer JL. Analysis of incomplete multivariate data. (1997) London, United Kingdom: Chapman and Hall Ltd.

CiteULike    Connotea    Del.icio.us    What's this?

Related articles in Am. J. Epidemiol.:

Invited Commentary: Fewell and Colleagues—Fuel for Debate

James Marshall
Am. J. Epidemiol. 2007 166: 656-658. [Abstract] [FREE Full Text]  

Fewell et al. Respond to "Fuel for Debate"

Zoe Fewell, George Davey Smith, and Jonathan A. C. Sterne
Am. J. Epidemiol. 2007 166: 659-661. [Extract] [FREE Full Text]  

This article has been cited by other articles:

				J. A.C. Delaney, B. E. Oddson, H. Kramer, S. Shea, B. M. Psaty, and R. L. McClelland Baseline Depressive Symptoms Are Not Associated With Clinically Important Levels of Incident Hypertension During Two Years of Follow-Up: The Multi-Ethnic Study of Atherosclerosis Hypertension, February 1, 2010; 55(2): 408 - 414. [Abstract] [Full Text] [PDF]

				S Collings, V Ivory, T Blakely, and J Atkinson Are neighbourhood social fragmentation and suicide associated in New Zealand? A national multilevel cohort study J Epidemiol Community Health, December 1, 2009; 63(12): 1035 - 1042. [Abstract] [Full Text] [PDF]

				B. Nosyk, Y. C. MacNab, H. Sun, B. Fischer, D. C. Marsh, M. T. Schechter, and A. H. Anis Proportional Hazards Frailty Models for Recurrent Methadone Maintenance Treatment Am. J. Epidemiol., September 15, 2009; 170(6): 783 - 792. [Abstract] [Full Text] [PDF]

				F de Vocht, H Kromhout, G Ferro, P Boffetta, and I Burstyn Bayesian modelling of lung cancer risk and bitumen fume exposure adjusted for unmeasured confounding by smoking Occup. Environ. Med., August 1, 2009; 66(8): 502 - 508. [Abstract] [Full Text] [PDF]

				J. Chevrier, B. Eskenazi, N. Holland, A. Bradman, and D. B. Barr Effects of Exposure to Polychlorinated Biphenyls and Organochlorine Pesticides on Thyroid Function during Pregnancy Am. J. Epidemiol., August 1, 2008; 168(3): 298 - 310. [Abstract] [Full Text] [PDF]

				P. Boffetta, J. K. McLaughlin, C. La Vecchia, R. E. Tarone, L. Lipworth, and W. J. Blot False-Positive Results in Cancer Epidemiology: A Plea for Epistemological Modesty J Natl Cancer Inst, July 16, 2008; 100(14): 988 - 995. [Abstract] [Full Text] [PDF]

				J. M. Robbins, G. E. Thatcher, D. A. Webb, and V. G. Valdmanis Nutritionist Visits, Diabetes Classes, and Hospitalization Rates and Charges: The Urban Diabetes Study Diabetes Care, April 1, 2008; 31(4): 655 - 660. [Abstract] [Full Text] [PDF]

				M. E. Andersen, J. L. Butenhoff, S.-C. Chang, D. G. Farrar, G. L. Kennedy Jr, C. Lau, G. W. Olsen, J. Seed, and K. B. Wallace Perfluoroalkyl Acids and Related Chemistries--Toxicokinetics and Modes of Action Toxicol. Sci., March 1, 2008; 102(1): 3 - 14. [Abstract] [Full Text] [PDF]

				E. A. Belongia, L. A. Coleman, J. G. Donahue, J. C. Nelson, M. L. Jackson, L. A. Jackson, L. Simonsen, C. Viboud, R. J. Taylor, M. M. Braun, et al. Effectiveness of Influenza Vaccination N. Engl. J. Med., December 27, 2007; 357(26): 2728 - 2731. [Full Text] [PDF]

				E. S. Iversen Jr, H. A. Katki, S. Chen, D. A. Berry, and G. Parmigiani Limited Family Structure and Breast Cancer Risk JAMA, November 7, 2007; 298(17): 2007 - 2007. [Full Text] [PDF]

				J. N. Weitzel, V. I. Lagos, and D. J. MacDonald Limited Family Structure and Breast Cancer Risk Reply JAMA, November 7, 2007; 298(17): 2007 - 2008. [Full Text] [PDF]

				J. Marshall Invited Commentary: Fewell and Colleagues Fuel for Debate Am. J. Epidemiol., September 15, 2007; 166(6): 656 - 658. [Abstract] [Full Text] [PDF]

				Z. Fewell, G. D. Smith, and J. A. C. Sterne Fewell et al. Respond to "Fuel for Debate" Am. J. Epidemiol., September 15, 2007; 166(6): 659 - 661. [Full Text] [PDF]

This Article Abstract FREE Full Text (PDF) All Versions of this Article: 166/6/646    most recent kwm165v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Related articles in Am. J. Epidemiol. Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Search for citing articles in: ISI Web of Science (27) Request Permissions Disclaimer Google Scholar Articles by Fewell, Z. Articles by Sterne, J. A. C. Search for Related Content PubMed PubMed Citation Articles by Fewell, Z. Articles by Sterne, J. A. C. Social Bookmarking          What's this?

Online ISSN 1476-6256 - Print ISSN 0002-9262

Copyright © 2010 Johns Hopkins Bloomberg School of Public Health

Oxford Journals Oxford University Press

• Site Map
• Privacy Policy
• Frequently Asked Questions

Other Oxford University Press sites: Oxford University Press Oxford Journals China Oxford Journals Japan American National Biography Booksellers' Information Service Children's Fiction and Poetry Children's Reference Corporate & Special Sales Dictionaries Dictionary of National Biography Digital Reference English Language Teaching Higher Education Textbooks Humanities International Education Unit Law Medicine Music Online Products Oxford English Dictionary Reference Rights and Permissions Science School Books Social Sciences Very Short Introductions World's Classics