Skip Navigation


American Journal of Epidemiology Advance Access originally published online on April 26, 2006
American Journal of Epidemiology 2006 164(1):63-68; doi:10.1093/aje/kwj155
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
164/1/63    most recent
kwj155v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (10)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Greenland, S.
Right arrow Articles by Gustafson, P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Greenland, S.
Right arrow Articles by Gustafson, P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

American Journal of Epidemiology Copyright © 2006 by the Johns Hopkins Bloomberg School of Public Health All rights reserved; printed in U.S.A.

Original Contribution

Accounting for Independent Nondifferential Misclassification Does Not Increase Certainty that an Observed Association Is in the Correct Direction

Sander Greenland1,2 and Paul Gustafson3

1 Department of Epidemiology, University of California, Los Angeles, CA
2 Department of Statistics, University of California, Los Angeles, CA
3 Department of Statistics, University of British Columbia, Vancouver, Canada

Correspondence to Dr. Sander Greenland, Departments of Epidemiology and Statistics, University of California, Los Angeles, CA 90095-1772 (e-mail: lesdomes{at}ucla.edu).

Received for publication May 28, 2005. Accepted for publication January 13, 2006.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 AN EXAMPLE
 DISCUSSION
 APPENDIX 1
 APPENDIX 2
 References
 
Researchers sometimes argue that their exposure-measurement errors are independent of other errors and are nondifferential with respect to disease, resulting in estimation bias toward the null. Among well-known problems with such arguments are that independence and nondifferentiality are harder to satisfy than ordinarily appreciated (e.g., because of correlation of errors in questionnaire items, and because of uncontrolled covariate effects on error rates); small violations of independence or nondifferentiality may lead to bias away from the null; and, if exposure is polytomous, the bias produced by independent nondifferential error is not always toward the null. The authors add to this list by showing that, in a 2 x 2 table (for which independent nondifferential error produces bias toward the null), accounting for independent nondifferential error does not reduce the p value even though it increases the point estimate. Thus, such accounting should not increase certainty that an association is present.

bias; epidemiologic methods; measurement error; misclassification; odds ratio; relative risk; validation

Abbreviations: IN, independent nondifferential; MLE, maximum likelihood estimate


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
AN EXAMPLE
 DISCUSSION
 APPENDIX 1
 APPENDIX 2
 References
 
Error in a measure X of an exposure T is independent of other errors and nondifferential with respect to other variables when the chance of a given error in X depends on only T. We will call such error IN error. It has long been known that IN error in a binary measure X of a binary T reduces the power of tests for the association of T with an outcome Y, and it induces a bias toward the null in common 2 x 2 table association measures (1Go, 2Go), apart from a few special cases in which no bias occurs (3Go). This fact has fueled intuitions that, given IN error, one can be more certain that an association is present than indicated by conventional statistics relating X to Y.

Unfortunately, such intuitions can be misleading. For example, if an association is present, IN error can produce bias away from the null or reverse a trend when T and X are polytomous or continuous (4Go, 5Go), and additional conditions are required to ensure that IN error does not bias an estimated trend past the null (6Go). Furthermore, the IN condition does not ensure that an observed (as opposed to expected) estimate is closer to the null than the true measure, since random error away from the null can sometimes exceed the bias toward the null (7Go, 8Go). Finally, in ecologic regression, the IN condition tends to lead to bias away from the null under the same conditions that it produces bias toward the null in the individual-level data (9Go).

We present yet another counterintuitive result: That in the simple case of a 2 x 2 table, in which IN error does produce bias toward the null, statistical adjustment for misclassification does not reduce the p value even though the adjusted point estimate moves away from the null (as intuition expects). Hence, the adjustment should not increase certainty (in either an informal or Bayesian sense) that an association is present. In this paper, we illustrate this phenomenon by using a real study, and we provide some intuition. Elsewhere, we discuss Bayesian analogues of the phenomenon (10Go).


   AN EXAMPLE
TOP
 ABSTRACT
 INTRODUCTION
 AN EXAMPLE
DISCUSSION
 APPENDIX 1
 APPENDIX 2
 References
 
Consider a case-control study of antibiotic use during pregnancy and subsequent occurrence of sudden infant death syndrome (11Go). Let Y be the indicator of sudden infant death syndrome and X be the indicator of a "yes" response to the antibiotic question. For 173 of 775 cases and 134 of 797 controls, X = 1. Superficially, the association appears unequivocal: The odds ratio is 1.42 with 95 percent confidence limits of 1.11, 1.83; the Wald p value (from taking the log odds ratio divided by its standard error) is 0.01, as are the likelihood-ratio and score (Pearson chi-squared) p values.

Of course, this positive association could be due to biases such as differential recall; nonetheless, given the small p values, it might seem that chance alone is not a reasonable explanation. Furthermore, chance plus IN error is not a reasonable explanation, because IN error does not harm the validity of the p value (i.e., under the null hypothesis that T and Y are unassociated, the p value for the X-Y association remains the probability that the test statistic is at least as large as observed provided that the error in X is IN) (1Go). Hence, it seems we should be confident in rejecting the T-Y null.

Adjustment with known classification rates
Suppose we examine the potential impact of IN error on these results by using some educated guesses about the errors in X as a measure of the true antibiotic-use indicator T—that is, conduct a sensitivity analysis ((12Go), chapter 19). Starting with sensitivity (chance of X = 1 given T = 1) and specificity (chance of X = 0 given T = 0) of 0.85, basic formulas to correct for IN error (appendix 1) yield an adjusted odds ratio estimate of 4.4, much larger than the unadjusted estimate of 1.4.

The dramatic increase (1.4 to 4.4) accords with intuition that IN error should have deflated the estimate, and it might make one think that a positive association is even more likely in light of IN error. Nonetheless, the adjusted Wald p value is now 0.06! This increase in p is a statistical artifact of the Wald method, however (13Go, 14Go), and the likelihood-ratio and score p values remain 0.01. Thus, even if we were 100 percent certain that sensitivity and specificity were 0.85 for both cases and controls, we should be no more confident that the T-Y association is positive than we were from examining only the unadjusted results (which corresponds to assuming no misclassification, so that the X-Y and T-Y associations are identical).

Of course, we are not certain about the error rates and so should at least try other pairs. Table 1 shows results for various choices. It is clear that specificity is most important (unsurprising given the low exposure prevalence) and that the estimates blow up as the specificity approaches 693/797 = 0.83 {approx} 5/6, the smallest value for which the adjustment formula yields positive odds ratios (Bayesian estimates do not blow up in this situation (15Go)). In every case, the odds ratio and confidence limits are larger after adjustment, yet the adjustment never decreases the p value. Thus, in this setting, adjusting for IN error should not increase our certainty that the T-Y association is positive, even though it shifts our estimates upward.


View this table:
[in this window]
[in a new window]
  		TABLE 1. Sensitivity analysis of a case-control study of antibiotic use (11, 29) obtained by questionnaire* and sudden infant death syndrome for independent nondifferential misclassification under various combinations of sensitivity and specificity of the questionnaire as a measure of actual use{dagger}

 
There is an intuitive reason for the latter phenomenon. The estimated variance of the log odds ratio is inversely proportional to the table numbers, hence it is most sensitive to the smallest numbers, and misclassification adjustment can make those numbers smaller. Thus, even though the log odds ratio estimate moves away from the null upon adjustment for IN error, the random variability of that estimate increases as well, so that the net certainty about the direction of the association does not increase. In fact, because the standard error increases even faster than the point estimate, the Wald p value increases upon adjustment, while the more accurate likelihood-ratio and score p values remain fixed (appendix 1). As a result, adjustment does not change our inference about the presence of a positive association, even when it changes the point and interval estimates dramatically (table 1).

Adjustment with validation data
Each of the analyses within a sensitivity analysis unrealistically assumes that the sensitivity and specificity are known exactly, leaving only random error unknown. The conventional analysis (no adjustment for misclassification) is but the one point in the sensitivity analysis that assumes (unrealistically) there was no classification error.

In reality, when important classification error is present, we never know its extent. Although often impractical because of ethics and cost, the most direct way to address this ignorance involves obtaining validation data on a subsample of subjects, as was done in the study of sudden infant death syndrome; table 2 shows the data. With these data, one can use adjustments more efficient than formulas based on only sensitivity and specificity estimates from those data (16Go–18Go). Given this efficiency gain, one might expect increased certainty after adjusting for IN error. Instead, from the whole-study data and those in table 2, the maximum likelihood estimate (MLE) of the T-Y odds ratio is 1.49 with 95 percent confidence limits of 1.02, 2.17; Wald p = 0.04. Thus, despite adding information through the validation data, the adjustment did not reduce uncertainty about the size and direction of the association (and in fact in this example appeared to leave even more uncertainty). More generally, a reviewer of this paper pointed out that the Wald p value obtained from regression-calibration adjustment (19Go) will not change with adjustment: Under the null hypothesis, the adjustment factors for the estimate and the standard error are equal and hence cancel out in the Wald statistic (estimate/standard error).


View this table:
[in this window]
[in a new window]
  		TABLE 2. Validation substudy of 211 cases and 217 controls from a study of sudden infant death syndrome (indicated by Y = 1) and antibiotic use (29)*

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 AN EXAMPLE
 DISCUSSION
APPENDIX 1
 APPENDIX 2
 References
 
Many expectations about the impact of IN error are not always borne out in reality: Bias can be away from the null (4Go, 5Go), and, even when the bias is toward the null (i.e., the estimate is too close to the null on average), a particular estimate may be an overestimate (7Go, 8Go). We have additionally shown that even in the simple 2 x 2 case, when the bias must be toward the null, allowance for IN error does not increase certainty that the association is in the observed direction.

In keeping with common practice, we have focused on the ordinary two-sided p values obtained from large-sample (approximate) chi-squared statistics. While we agree that p values are terribly overused and misinterpreted, one-sided p values can often be translated into Bayesian inferences (20Go, 21Go). For example, if the prior distribution is sufficiently diffuse, in typical asymptotically normal examples, the smaller of the two one-sided p values (i.e., half the two-sided p value) approximates the posterior probability that the observed association is in the wrong direction (appendix 2). Thus, although the ordinary two-sided p value is not (as often misinterpreted) the probability that the null is true, half that p value often approximates the certainty that the true association is in the direction opposite of that observed and thus measures how unsure we should be that our result is qualitatively correct.

The independence condition is often overlooked, and common claims that nondifferentiality alone (equal error rates among cases and controls) produces bias toward the null are simply wrong; dependent errors can easily produce bias away from the null even when the errors are nondifferential and the null is true (22Go, 23Go). Furthermore, departures from nondifferentiality can easily arise even with prospectively collected data (24Go, 25Go), and small departures can produce bias away from the null when noncase specificity minus case specificity approaches the true exposure prevalence (26Go). Finally, IN error in a binary confounder Z results in bias away from the null in the Z-adjusted estimate if the direction of confounding by Z is away from the null (27Go), although this is not always so for polytomous confounders (5Go, 28Go).

We have focused on IN error because its recognition provokes expectations of "conservative" results. These expectations are often unwarranted with regard to inference about the presence or direction of an association. Recognition of differential error can also produce misleading expectations. In the above example, there is some basis to expect recall bias (higher sensitivity and/or lower specificity among cases) (29Go), which can lead to bias away from the null, although the data are borderline compatible with nondifferentiality. It seems a common belief that the presence of such differential recall implies that the observed estimate is an overestimate. This is not so. Although the differential portion of the error does produce an upward bias in a positive association, there remains a downward bias due to shared error components; this downward bias may more than counterbalance the recall bias, so that the average estimate understates the true association and the test statistic is conservative (30Go). In addition, even when the net bias is upward, downward random error might more than counterbalance the upward bias, so that the observed estimate could still be an underestimate.

Recent literature has recommended extending one-pair-at-a-time adjustment by placing prior distributions on sensitivities and specificities, drawing random values from these distributions, adjusting the results by using these values, and summarizing the distribution of adjusted results (31Go–34Go). Under certain assumptions, these distributions can approximate Bayesian posterior distributions based on the same priors (35Go). Alternatively, one can just use the prior distributions directly in a Bayesian analysis (36Go). Doing so, we have found that, given IN error and informative priors for all parameters, the posterior probability that the observed association is in the correct direction can actually be reduced by adjustment (10Go).

Collection of validation data, while laudable, is no panacea for the problems reviewed above. First, those data may be unethical or prohibitively expensive to collect, and the amount collected may be so small that they leave tremendous uncertainty about the relation of X to the truth T and hence of T to Y. Second, those data often add only another presumably better measure S of T (an "alloyed gold standard" S). Most medical records provide prescription data, not compliance, and noncompliance leads to imperfect specificity of the record. Diet and nutrition validation add only food-diary data, which is still far from life history, and for which errors are likely dependent. When S is imperfect, adjustments based on assuming that S is perfect (S = T) may introduce more bias than they remove (37Go). Proper adjustment must account for the error in S as well as in X as a measure of T (38Go), and this adjustment will not increase certainty about the direction and size of the association, despite bias toward the null.

In summary, the IN condition is difficult to satisfy. Even when satisfied, it does not always guarantee bias toward the null; even when the bias it produces is toward the null and adjustment moves the estimates away from the null, it does not follow that adjusting for the error will strengthen the evidence that the observed association is in the correct direction.


   APPENDIX 1
TOP
 ABSTRACT
 INTRODUCTION
 AN EXAMPLE
 DISCUSSION
 APPENDIX 1
APPENDIX 2
 References
 
Adjustment Formulas
Let qxt = P(X = x|T = t), Q = [qxt] in order 1,0, and let axy be the number of subjects with X = x, Y = y; then, A = [axy] is the X-Y table of counts with E(A) = EA = QEC, where EC is the expected correctly classified T-Y table. Suppose Q is known with |Q| = det(Q) = q11q00 – q10q01 > 0, but there is no information on EC except through A and Q. A "corrected" table of T-Y counts is C = [cty] = Q–1A (39Go). With c = vec(C), the Q-adjusted log odds ratio from C is g(c) = ln(c11c00/c10c01) with asymptotic variance estimate

Formula 1(1)
(16Go), which shows the inverse dependence on the corrected cell counts. Under perfect classification (X = T, equivalently Q = I, and hence C = A), vg reduces to the usual estimate {Sigma}xy(1/axy); as the classification worsens, however, this estimate becomes highly inflated.

More generally and formally, let a be a vector of observed counts related to a latent mean vector µc by µa {equiv} E(a) = Qµc, where Q is a known invertible matrix. Then, for c {equiv} Q–1a, E(c) = µc. The unconstrained Poisson log-likelihood (LL) is

Formula 2(2)
with score equation

Formula 3(3)
where Va {equiv} diag(µa); c is the solution to equation 3. The expected information is –Formula 3 hence, an asymptotic covariance matrix for c is Vc {equiv} Q–1VaQ'–1.

Returning to the special case of a 2 x 2 table with nondifferential X (across-column) misclassification only, a = vec(A), µa {equiv} vec(EA) = E(a), µc {equiv} vec(EC), and Q is block diagonal with equal 2 x 2 blocks Q. Because c is the MLE of µc, g(c) is the MLE of g(µc); similarly, a = Qc is the unconstrained MLE of µa. Because

Formula 4(4)
an asymptotic variance {nu}g for g(c) is dg'Vcdg; substitution of a and c for µa and µc yields vg (equation 1). Similar algebra under multinomial and two-binomial models for the observed table A yields identical results for g(c) and {nu}g, paralleling the relation of log-linear-model likelihoods under Poisson, multinomial, and product-binomial sampling in ordinary contingency tables (40Go).

Now, c+y = a+y and similarly µc+y = µa+y, independent of Q (reflecting the fact that there is no misclassification across rows, y); furthermore, Q[µc1+ µc0+]' = QEC1 = EA1 = [µa1+ µa0+]'.

Now, under the null hypothesis, EC = [µc1+ µc0+]'[µc+1 µc+0]/µ++, so EA = QEC = Q[µc1+ µc0+]'[µc+1 µc+0]/µ++ = [µa1+ µa0+]'[µa+1 µa+0]/µ++. The observed margins ax+, a+y are the sufficient statistics and hence the MLEs for their expectations µax+ and µa+y. Thus, the null-constrained MLE of EA is [a1+ a0+]'[a+1 a+0]/a++ = [exy], where the exy = ax+a+y/a++ do not depend on Q, and hence neither do the likelihood-ratio or score statistics 2{Sigma}xyaxyln(axy/exy) and a++ 3(a11 – e11)2/(a1+a+1a0+a+0). The latter is the usual Pearson statistic, whose invariance under Q is a corollary of algebra first given by Bross (1Go). This result also follows by substituting vec([µc1+ µc0+]'[µc+1 µc+0]) for µc in the likelihood (equation 2) and maximizing.

Next, suppose Q is unknown but internal validation data are available, as in table 2. Then, the MLE of the odds ratio follows from maximum likelihood for double sampling in Espeland and Hui (41Go) or from methods for estimating cell probabilities from contingency tables with missing data (42Go). Under IN error, however, this estimate has no closed form (18Go). The example results were computed by using both expectation-maximization and Gauss-Newton procedures for maximization of the incomplete-data likelihood (42Go). Since these computations now involve maximizing over the elements in Q, the likelihood-ratio statistic is no longer identical to that obtained when Q is known.


   APPENDIX 2
TOP
 ABSTRACT
 INTRODUCTION
 AN EXAMPLE
 DISCUSSION
 APPENDIX 1
 APPENDIX 2
References
 
One-sided p Values as Posterior Probabilities
Consider an approximately normally distributed efficient unbiased estimator T of {Theta} with consistent variance estimator V. An approximate (local) two-sided p value for testing {Theta} = {theta} upon observing T = t > {theta} and V = v is P = P(Z > |t – {theta}|/v1/2), where Z is a standard normal variate, and P/2 = P{Z > (t – {theta})/v1/2} = c{int}(t,{infty})exp{–(T – {theta})2/2v}dT, where c = (2{pi}v)–1/2. By symmetry of the density, we get P/2 = c{int}(–{infty},{theta})exp{–(T – t)2/2v}dT, and, upon change of variable from T to {Theta}, we get P/2 = c{int}(–{infty},{theta})exp{–({Theta} – t)2/2v}d{Theta}.

Now, under an improper uniform prior on {Theta}, the posterior density for {Theta} is approximately normal(t,v). The last integral can thus be seen as an approximate posterior probability of {Theta} < {theta}; because t > {theta}, this is the posterior probability that t is on the wrong side of {theta}. Parallel arguments show that P/2 is the posterior probability of {Theta} > {theta} when t < {theta}. Refer to Leonard and Hsu ((43Go), section 3.6) for parallel relations of confidence intervals to posterior intervals.


   ACKNOWLEDGMENTS
 
The authors thank Dr. Katherine J. Hoggatt for helpful comments.

Conflict of interest: none declared.


   References
TOP
 ABSTRACT
 INTRODUCTION
 AN EXAMPLE
 DISCUSSION
 APPENDIX 1
 APPENDIX 2
 References
 

  1. Bross I. Misclassification in 2 x 2 tables. Biometrics 1954;10:478–86.[CrossRef][Web of Science]
  2. Newell DJ. Errors in the interpretation of errors in epidemiology. Am J Public Health 1962;52:1925–8.[Free Full Text]
  3. Poole C. Nondifferential misclassification does not always produce bias toward the null. (Abstract). Am J Epidemiol 1985;118:508.
  4. Dosemeci M, Wacholder S, Lubin JH. Does nondifferential misclassification of exposure always bias a true effect toward the null value? Am J Epidemiol 1990;132:746–8.[Abstract/Free Full Text]
  5. Wacholder S. When measurement errors correlate with truth: surprising effects of nondifferential misclassification. Epidemiology 1995;6:157–61.[Web of Science][Medline]
  6. Weinberg CR, Umbach DM, Greenland S. When will nondifferential misclassification of an exposure preserve the direction of a trend? Am J Epidemiol 1994;140:565–71.[Abstract/Free Full Text]
  7. Thomas DC. Re: "When will nondifferential misclassification of an exposure preserve the direction of a trend?" (Letter). Am J Epidemiol 1995;142:782–4.[Free Full Text]
  8. Jurek AM, Greenland S, Maldonado GM, et al. Proper interpretation of nondifferential misclassification effects: expectations versus observations. Int J Epidemiol 2005;34:680–7.[Abstract/Free Full Text]
  9. Brenner H, Savitz DA, Jöckel KH, et al. The effects of nondifferential misclassification in ecologic studies. Epidemiology 1992;3:85–95.
  10. Gustafson P, Greenland S. Curious phenomena in Bayesian adjustment for exposure misclassification. Stat Med 2006;25:87–103.[CrossRef][Web of Science][Medline]
  11. Kraus JF, Greenland S, Bulterys MG. Risk factors for sudden infant death syndrome in the U.S. Collaborative Perinatal Project. Int J Epidemiol 1989;18:113–20.[Abstract/Free Full Text]
  12. Rothman KJ, Greenland S. Modern epidemiology. 2nd ed. Philadelphia, PA: Lippincott, 1998.
  13. Hauck WW, Donner A. Wald's test as applied to hypotheses in logit analysis. J Am Stat Assoc 1977;72:851–3. Corrigendum 1980;75:482.[CrossRef][Web of Science]
  14. Vaeth M. On the use of Wald's test in exponential families. Int Stat Rev 1985;54:199–214.
  15. Gustafson P, Le ND, Saskin R. Case-control analysis with partial knowledge of exposure misclassification probabilities. Biometrics 2001;57:598–609.[CrossRef][Web of Science][Medline]
  16. Greenland S. Variance estimation for epidemiologic effect estimates under misclassification. Stat Med 1988;7:745–57.[Web of Science][Medline]
  17. Marshall RJ. Validation study methods for estimating exposure proportions and odds ratios with misclassified data. J Clin Epidemiol 1990;43:941–7.[CrossRef][Web of Science][Medline]
  18. Lyles RH. A note on estimating crude odds ratios in case-control studies with differentially misclassified exposure. Biometrics 2002;58:1034–7.[CrossRef][Web of Science][Medline]
  19. Rosner B, Willett WC, Spiegelman D. Correction of logistic regression relative risk estimates and confidence intervals for systematic within-person measurement error. Stat Med 1989;8:1051–70.[Web of Science][Medline]
  20. Casella G, Berger RL. Reconciling Bayesian and frequentist evidence in the one-sided testing problem. J Am Stat Assoc 1987;82:106–11.[CrossRef][Web of Science]
  21. Berger J. Could Fisher, Jeffreys and Neyman have agreed on testing? (with discussion). Stat Sci 2003;18:1–32.[CrossRef][Web of Science]
  22. Chavance M, Dellatolas G, Lellouch J. Correlated nondifferential misclassifications of disease and exposure: application to a cross-sectional study of the relation between handedness and immune disorders. Int J Epidemiol 1992;21:537–46.[Abstract/Free Full Text]
  23. Kristensen P. Bias from nondifferential but dependent misclassification of exposure and outcome. Epidemiology 1992;3:210–15.[Web of Science][Medline]
  24. Flegal KM, Keyl PM, Nieto FJ. Differential misclassification arising from nondifferential errors in exposure measurement. Am J Epidemiol 1991;134:1233–44.[Abstract/Free Full Text]
  25. Wacholder S, Dosemeci M, Lubin JH. Blind assignment of exposure does not always prevent nondifferential misclassification. Am J Epidemiol 1991;134:433–7.[Abstract/Free Full Text]
  26. Maldonado G, Greenland S, Phillips C. Approximately nondifferential exposure misclassification does not ensure bias toward the null. (Abstract). Am J Epidemiol 2000;151:S39.
  27. Greenland S. The effect of misclassification in the presence of covariates. Am J Epidemiol 1980;112:564–9.[Abstract/Free Full Text]
  28. Brenner H. Bias due to nondifferential misclassification of a polytomous confounder. J Clin Epidemiol 1993;46:57–63.[CrossRef][Web of Science][Medline]
  29. Dres C, Kraus JF, Greenland S. Recall bias in a case-control study of sudden infant death syndrome. Int J Epidemiol 1990;19:405–11.[Abstract/Free Full Text]
  30. Drews C, Greenland S. The impact of differential recall on the results of case-control studies. Int J Epidemiol 1990;19:1107–12.[Abstract/Free Full Text]
  31. Lash TL, Fink AK. Semi-automated sensitivity analysis to assess systematic errors in observational epidemiologic data. Epidemiology 2003;14:451–8.[Web of Science][Medline]
  32. Phillips CV. Quantifying and reporting uncertainty from systematic errors. Epidemiology 2003;14:459–66.[Web of Science][Medline]
  33. Greenland S. The impact of prior distributions for uncontrolled confounding and response bias: a case study of the relation of wire codes and magnetic fields to childhood leukemia. J Am Stat Assoc 2003;98:47–54.[CrossRef][Web of Science]
  34. Fox MP, Lash TL, Greenland S. A method to automate probabilistic sensitivity analyses of misclassified binary variables. Int J Epidemiol 2005;34:1370–7.[Abstract/Free Full Text]
  35. Greenland S. Multiple-bias modeling for observational studies (with discussion). J R Stat Soc (A) 2005;168:267–308.[CrossRef]
  36. Gustafson P. Measurement error and misclassification in statistics and epidemiology. Boca Raton, FL: Chapman and Hall, 2003.
  37. Wacholder S, Armstrong B, Hartge P. Validation studies using an alloyed gold standard. Am J Epidemiol 1993;137:1251–8.[Abstract/Free Full Text]
  38. Carroll RJ, Ruppert D, Stefanski LA. Measurement error in nonlinear models. Boca Raton, FL: Chapman and Hall, 1995.
  39. Barron BA. The effects of misclassification on the estimation of relative risk. Biometrics 1977;33:414–18.[CrossRef][Web of Science][Medline]
  40. Bishop YMM, Fienberg SE, Holland PW. Discrete multivariate analysis: theory and practice. Cambridge, MA: MIT Press, 1975.
  41. Espeland M, Hui SL. A general approach to analyzing epidemiologic data that contain misclassification errors. Biometrics 1987;43:1001–12.[CrossRef][Web of Science][Medline]
  42. Little RJA, Rubin DB. Statistical analysis with missing data. 2nd ed. New York, NY: Wiley, 2002.
  43. Leonard T, Hsu JSJ. Bayesian methods: an analysis for statisticians and interdisciplinary researchers. New York, NY: Cambridge University Press, 1999.

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Int J EpidemiolHome page
M. Leu, K. Czene, and M. Reilly
Bias correction of estimates of familial risk from population-based cohort studies
Int. J. Epidemiol., October 13, 2009; (2009) dyp304v1.
[Abstract] [Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
P. Cummings
Propensity Scores
Arch Pediatr Adolesc Med, August 1, 2008; 162(8): 734 - 737.
[Full Text] [PDF]

Home page
 J. Epidemiol. Community HealthHome page
G Maldonado
Adjusting a relative-risk estimate for study imperfections
J Epidemiol Community Health, July 1, 2008; 62(7): 655 - 663.
[Abstract] [Full Text] [PDF]

Home page
 Int J EpidemiolHome page
A. M Jurek, S. Greenland, and G. Maldonado
Brief Report: How far from non-differential does exposure or disease misclassification have to be to bias measures of association away from the null?
Int. J. Epidemiol., April 1, 2008; 37(2): 382 - 385.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
J. M. Oakes and T. R. Church
Invited Commentary: Advancing Propensity Score Methods in Epidemiology
Am. J. Epidemiol., May 15, 2007; 165(10): 1119 - 1121.
[Abstract] [Full Text] [PDF]

Home page
 Int J EpidemiolHome page
S. GREENLAND, M. P FOX, and T. L LASH
Reply to Roger Marshall
Int. J. Epidemiol., December 1, 2006; 35(6): 1589 - 1590.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
164/1/63    most recent
kwj155v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (10)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Greenland, S.
Right arrow Articles by Gustafson, P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Greenland, S.
Right arrow Articles by Gustafson, P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?