American Journal of Epidemiology

American Journal of Epidemiology Advance Access originally published online on November 14, 2008
American Journal of Epidemiology 2009 169(3):267-270; doi:10.1093/aje/kwn362

This Article Abstract FREE Full Text (PDF) All Versions of this Article: 169/3/267    most recent kwn362v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Related articles in Am. J. Epidemiol. Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Search for citing articles in: ISI Web of Science (2) Request Permissions Disclaimer Google Scholar Articles by Gefeller, O. Search for Related Content PubMed PubMed Citation Articles by Gefeller, O. Social Bookmarking          What's this?

American Journal of Epidemiology © The Author 2008. Published by the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

Invited Commentary

Invited Commentary: Recall Bias in Melanoma—Much Ado About Almost Nothing?

Olaf Gefeller

Correspondence to Dr. Olaf Gefeller, Department of Medical Informatics, Biometry, and Epidemiology, University of Erlangen-Nuremberg, Waldstr. 6, D-91054 Erlangen, Germany (e-mail: gefeller{at}rzmail.uni-erlangen.de).

Received for publication August 14, 2008. Accepted for publication August 28, 2008.

	ABSTRACT

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 
Recall bias has been given considerable attention in textbooks and methodological research because of its potential to jeopardize the validity of epidemiologic results. Case-control studies on self-reported ultraviolet radiation exposure as a risk factor for melanoma have been described as especially prone to the deleterious effect of recall bias because of the growing public awareness about these risks. Using an ideal test-retest design in a large nested case-control study, Parr et al. (Am J Epidemiol. 2009;169(3):257–266) examined to what extent recall bias in melanoma risk factors is actually identifiable and which consequences its presence has on effect estimates of these risk factors. They found only minor indications of recall bias, showing an inconsistent overall pattern and a quite negligible effect on risk estimates. Recall bias was not observed in those exposures where it was most expected (solarium use and other ultraviolet radiation-related exposures). Their findings cannot be used as an argument that future case-control studies in melanoma epidemiology should be avoided because of the biasing effect of recall bias.

bias (epidemiology); dermatology; epidemiologic methods; melanoma; ultraviolet rays

---

Abbreviations: NOWAC, Norwegian Women and Cancer Study

	INTRODUCTION

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 
The evaluation of potential bias is a key topic in epidemiology. All observational research is prone to lots of sources of bias that can seriously distort the truth. Epidemiologic textbooks and courses devote tremendous efforts to sensitize the novice about the problem of bias. Typically, at the end of a classic epidemiologic education, there are 2 types of students: One does not believe any empirical finding from an observational study and is afraid that one of the dozens of sources of potential bias has produced spurious associations or has masked a true exposure-disease-relation; the other type of student gets quickly rid of the alleged ballast of theoretical discussions about bias and focuses on analyzing epidemiologic data, hopefully implementing techniques to control for bias whenever possible.

	RECALL BIAS IN GENERAL

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 
One specific form of bias that has been given considerable attention in nearly all traditional and modern epidemiologic textbooks, however, does not belong to those forms of bias that can be easily taken into account during analysis. Its presence in retrospective research was recognized quite early. A classic example is given in Social Origins of Depression: A Study of Psychiatric Disorders in Women by Brown and Harris (1), who describe a study by Stott in 1958 (2). Stott compared data from interviews with mothers of children with Down syndrome and mothers of normal children. He found that, in the first group, significantly more shocks during pregnancy had been reported by the mothers and concluded that socioemotional factors played an etiologic role in the disease process. Later research, of course, clarified that chromosomal abnormalities were the cause, and mothers of children with Down syndrome probably redefined ordinary events as shocks in an effort to explain what had happened.

Such a differential memory of exposure history in the study subgroups to be compared has often been termed "recall bias." It is important to note that recall bias is not equivalent to memory failure itself. If an incorrect memory of past exposure, which is inevitable to some degree in all retrospective studies, affects the groups to be compared to an identical extent, recall bias will not occur. This form of bias is a special variant of an information bias that can be thought of as a form of differential misclassification. It can arise in all studies using retrospective exposure ascertainment when study subjects are aware of their disease status and show a (intentional or unintentional) tendency to report past exposure differentially depending on their disease status. Both downplaying and exaggeration of individual exposure have been offered as the typical reaction pattern of diseased study subjects when recollecting their exposure history. In epidemiologic practice, such a situation may occur in many case-control studies.

Several methodological papers (3–7) have addressed this form of bias and analyzed its consequences, but no clear-cut message could be drawn from these analyses. The direction and magnitude of the bias depend in a complex way on quite a number of parameters of the specific study framework. Recall bias, like differential misclassification in general, is, therefore, the nasty type of bias in epidemiology: Everyone knows that it might be present, but usually nobody knows whether this is actually the case and what consequences its presence will have.

Despite the attention given to recall bias in epidemiologic textbooks and methodological papers, empirical research focusing on recall bias itself is quite limited. Only scattered reports trying to assess the presence, direction, and magnitude of recall bias in specific settings can be found in the literature. There is much more speculation in a lot of modern case-control studies about potential recall bias than sound data corroborating its existence and delineating its effect on the study results. The reason for this discrepancy is simple: It is quite difficult to study recall bias in practice. Strictly speaking, the only methodologically appropriate approach consists of a test-retest design in cases and controls where the first assessment had been made prior to the diagnosis of the cases and the second assessment after the diagnosis (the controls have to be studied to disentangle recall bias from nondifferential misclassification that can also show some temporal trend in exposure reporting).

	RECALL BIAS IN MELANOMA

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 
Not all areas of epidemiologic research are considered to be equally prone to the problem of recall bias. When exposure information has to be retrospectively ascertained over a wide-stretching time period and can be based solely on self-reports of study subjects without the opportunity of external verification, recall bias is regarded as a serious threat to the validity of the corresponding studies. Such concern has especially been expressed for those exposures that attract public attention as the knowledge of a harmful effect of an exposure is a prerequisite for study subjects to change their reporting behavior of individual exposure. In melanoma epidemiology, examining those risk factors related to ultraviolet radiation exposure in case-control studies is thus notably problematic with respect to the presence of recall bias, since all the conditions given above hold for these factors.

The established effect of ultraviolet radiation exposure on the occurrence of melanoma (8) has received intensive media coverage during the last 2 decades. Gandini et al. (9) observed in their meta-analysis on the effect of total sun exposure significant heterogeneity between effect estimates published before and after 1990 and attributed this finding to a growing impact of recall bias. They also explained discrepant results from population-based and hospital-based case-control studies by the presence of recall bias when hospital-based control groups were interviewed. They argued that controls with dermatologic diseases—constituting the majority in published hospital-based case-control studies of melanoma—may be more aware of the general effect of ultraviolet radiation than population controls are (9).

Public awareness resulting in recall bias has also been made responsible for the lack of an effect of ultraviolet radiation exposure and solarium use on melanoma risk in a recent large European case-control study (10). In a separately published additional analysis, evidence has been presented that recall bias is a plausible explanation for the phenomena observed in that study (11).

Contrary to the many investigations evaluating reproducibility of information in melanoma risk factors (12–21), only a few attempts to study recall bias on melanoma empirically can be found in the literature (22–27). These findings have been briefly summarized in the work by Pfahlberg and Gefeller (28) and in the Discussion section of Parr et al. (29) in this issue of the Journal. All of these previous investigations suffer from limitations: very limited sample size (25, 26), slight changes in the wording of questions in the first and second exposure ascertainments (23, 27), or indirect approaches not using a test-retest design (22, 24).

	RECALL BIAS IN MELANOMA BASED ON THE NORWEGIAN WOMEN AND CANCER STUDY

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 
This unsatisfactory situation is changed by the publication of the sound work by Parr et al. (29). Based on the Norwegian Women and Cancer Study (NOWAC) cohort, a nested case-control study of 162 melanoma cases and 1,242 matched controls was constructed to analyze recall bias in an ideal setting. Comprehensive exposure data had been ascertained at enrollment and were regathered several years later in 2004. Then, Parr et al. compared shifts in responses between melanoma cases and controls, finding little evidence of a strong differential recall of exposure history. They observed some changes in reported exposure to melanoma risk factors between the first and second assessments, but these changes occurred in cases and controls. A systematic shift in cases only was confined to the variable "skin color after chronic sun exposure," a finding that is hard to interpret. In addition, a significantly higher number of nevi in the second assessment was reported by cases and controls with the magnitude of shift being larger for cases than for controls. For all other risk factors, especially for the interesting ultraviolet radiation-related exposures, no evidence of recall bias could be found. Parr et al. conclude that "indications of recall bias were found ..., but ... the results were inconsistent for the different exposures" (29, p. 257), which correctly summarizes the surprising pattern of results. Recall bias was not found in those exposures where it was most expected (solarium use and other ultraviolet radiation-related exposures) and, where it was found, it cannot be easily interpreted.

In addition to presenting these data on changes in exposure recall, Parr et al. (29) also analyzed whether the use of exposure information from the first (prospective) assessment leads to different risk estimates than the use from the second (retrospective) exposure assessment. This additional analysis addresses the magnitude of the effect of recall bias on study findings when using such imperfect exposure data. Overall, the differences between the results were quite negligible. The same risk factors were identified in both approaches. Although the estimates of different risk categories showed some fluctuations, no consistent pattern of the discrepancies could be identified. A limitation of this analysis was, however, that the risk estimates in some strata of the exposure variables were quite imprecise because of the low number of cases in these categories. Nevertheless, even an indication that the use of retrospectively gathered exposure data will produce different conclusions on melanoma risk factors was not observed.

	WHAT SHOULD BE DONE IN THE FUTURE?

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 
NOWAC offers an ideal framework for studying melanoma risk factors in a prospective manner. The first analysis published in 2003 (30) suffered, however, from a low number of only 187 melanoma cases, limiting a more detailed evaluation of risk profiles. A second analysis after an extended follow-up period has been announced (31). A continuation of efforts to use the NOWAC framework for future studies is important for melanoma epidemiology and therefore strongly encouraged. This cohort study approach has the great advantage that nobody will criticize the findings using the argument that recall bias may have influenced the results. A repetition of the current methodological substudy on recall bias comprising a larger case group to raise the statistical power is also highly welcome to sort out myths from facts in this area.

The case-control approach has, however, been the dominating design in melanoma epidemiology for decades. Hundreds of such studies have assembled our current knowledge about the factors influencing melanoma risk. Despite all methodological shortcomings of the approach, the results of these investigations have guided us well to derive strategies for primary prevention of melanoma. Recently, concerns have been raised about whether case-control studies can still be used in this area because of potential recall bias resulting from the growing public awareness about melanoma risk factors (11). I am not sure whether the findings of the present study in this issue of the Journal will end this epidemiologic discussion. Recall bias is the nightmare of all epidemiologists conducting case-control studies and will remain a hot topic of controversial discussion in the melanoma community. However, some nightmares suddenly lose much of their scare if one awakes in the morning and realizes that all was just a bad dream. In some way, the data by Parr et al. (29) offer such a cozy morning feeling, as they show that the extent of recall bias is much lower than generally expected and that the consequences for the analysis of melanoma risk factors are minor. Definitely, their findings cannot be used as an argument that future case-control studies in melanoma epidemiology should be avoided because of the biasing effect of recall bias.

	ACKNOWLEDGMENTS

 
Author affiliation: Department of Medical Informatics, Biometry, and Epidemiology, University of Erlangen-Nuremberg, Erlangen, Germany.

This work was supported in part by a grant from the German Research Foundation (DFG Ge 637/4-4).

Conflict of interest: none declared.

	References

TOP ABSTRACT INTRODUCTION RECALL BIAS IN GENERAL RECALL BIAS IN MELANOMA RECALL BIAS IN MELANOMA... WHAT SHOULD BE DONE... References

 

1. Brown GW, Harris T. Social Origins of Depression: A Study of Psychiatric Disorders in Women (1978) New York, NY: Free Press.
2. Stott DH. Some psychosomatic aspects of casualty in reproduction. J Psychosom Res. (1958) 3(1):42–55.[Medline]
3. Barry D. Differential recall bias and spurious associations in case/control studies. Stat Med (1996) 15(23):2603–2616.[CrossRef][Medline]
4. Barry D, Livingstone V. The investigation and correction of recall bias for an ordinal response in a case-control study. Stat Med (2006) 25(6):965–975.[Medline]
5. Chouinard E, Walter S. Recall bias in case-control studies: an empirical analysis and theoretical framework. J Clin Epidemiol (1995) 48(2):245–254.[CrossRef][Web of Science][Medline]
6. Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol (1990) 43(1):87–91.[CrossRef][Web of Science][Medline]
7. Drews CD, Greeland S. The impact of differential recall on the results of case-control studies. Int J Epidemiol (1990) 19(4):1107–1112.[Abstract/Free Full Text]
8. Elwood JM, Jopson J. Melanoma and sun exposure: an overview of published studies. Int J Cancer (1997) 73(2):198–203.[CrossRef][Web of Science][Medline]
9. Gandini S, Sera F, Cattaruzza MS, et al. Meta-analysis of risk factors for cutaneous melanoma. II. Sun exposure. Eur J Cancer (2005) 41(1):45–60.[CrossRef][Web of Science][Medline]
10. Bataille V, Boniol M, De Vries E, et al. A multicentre epidemiological study on sunbed use and cutaneous melanoma in Europe. Eur J Cancer (2005) 41(14):2141–2149.[CrossRef][Web of Science][Medline]
11. de Vries E, Boniol M, Severi G, et al. Public awareness about risk factors could pose problems for case-control studies: the example of sunbed use and cutaneous melanoma. Eur J Cancer (2005) 41(14):2150–2154.[CrossRef][Web of Science][Medline]
12. Beane Freeman LE, Dennis LK, Lynch CF, et al. Test-retest of self-reported exposure to artificial tanning devices, self-tanning creams, and sun sensitivity showed consistency. J Clin Epidemiol (2005) 58(4):430–432.[CrossRef][Web of Science][Medline]
13. Berwick M, Chen YT. Reliability of reported sunburn history in a case-control study of cutaneous malignant melanoma. Am J Epidemiol (1995) 141(11):1033–1037.[Abstract/Free Full Text]
14. Bränström R, Kristjansson S, Ullén H, et al. Stability of questionnaire items measuring behaviours, attitudes and stages of change related to sun exposure. Melanoma Res. (2002) 12(5):513–519.[CrossRef][Web of Science][Medline]
15. English DR, Armstrong BK, Kricker A. Reproducibility of reported measurements of sun exposure in a case-control study. Cancer Epidemiol Biomarkers Prev (1998) 7(10):857–863.[Abstract]
16. Glanz K, Schoenfeld E, Weinstock MA, et al. Development and reliability of a brief skin cancer risk assessment tool. Cancer Detect Prev (2003) 27(4):311–315.[CrossRef][Web of Science][Medline]
17. Nadalin V, Bentvelsen K, Kreiger N. Reliability of self-reports: data from the Canadian Multi-Centre Osteoporosis Study (CaMos). Chronic Dis Can (2004) 25(2):28–31.[Medline]
18. Rosso S, Miñarro R, Schraub S, et al. Reproducibility of skin characteristic measurements and reported sun exposure history. Int J Epidemiol (2002) 31(2):439–446.[Abstract/Free Full Text]
19. van der Mei IA, Blizzard L, Ponsonby AL, et al. Validity and reliability of adult recall of past sun exposure in a case-control study of multiple sclerosis. Cancer Epidemiol Biomarkers Prev (2006) 15(8):1538–1544.[Abstract/Free Full Text]
20. Veierød MB, Parr CL, Lund E, et al. Reproducibility of self-reported melanoma risk factors in a large cohort study of Norwegian women. Melanoma Res. (2008) 18(1):1–9.
21. Westerdahl J, Anderson H, Olsson H, et al. Reproducibility of a self-administered questionnaire for assessment of melanoma risk. Int J Epidemiol (1996) 25(2):245–251.[Abstract/Free Full Text]
22. Cockburn M, Hamilton A, Mack T. Recall bias in self-reported melanoma risk factors. Am J Epidemiol (2001) 153(10):1021–1026.[Abstract/Free Full Text]
23. Han J, Colditz GA, Hunter DJ. Risk factors for skin cancers: a nested case-control study within the Nurses’ Health Study. Int J Epidemiol (2006) 35(6):1514–1521.[Abstract/Free Full Text]
24. Relova AS, Marrett LD, Klar N, et al. Predictors of self-reported confidence ratings for adult recall of early life sun exposure. Am J Epidemiol (2005) 162(2):183–192.[Abstract/Free Full Text]
25. Walter SD, Marrett LD, From L, et al. The association of cutaneous malignant melanoma with the use of sunbeds and sunlamps. Am J Epidemiol (1990) 131(2):232–243.[Abstract/Free Full Text]
26. Walter SD, Marrett LD, Shannon HS, et al. The association of cutaneous malignant melanoma and fluorescent light exposure. Am J Epidemiol (1992) 135(7):749–762.[Abstract/Free Full Text]
27. Weinstock MA, Colditz GA, Willett WC, et al. Recall (report) bias and reliability in the retrospective assessment of melanoma risk. Am J Epidemiol (1991) 133(3):240–245.[Abstract/Free Full Text]
28. Pfahlberg AB, Gefeller O. Errors in assessing risk factors for melanoma: lack of reproducibility is the minor problem. Melanoma Res. (2008) 18(4):300–301.[Medline]
29. Parr CL, Hjartåker A, Laake P, et al. Recall bias in melanoma risk factors and measurement error effects: a nested case-control study within the Norwegian Women and Cancer Study. Am J Epidemiol (2009) 169. (3):257–266.
30. Veierød MB, Weiderpass E, Thörn M, et al. A prospective study of pigmentation, sun exposure, and risk of cutaneous malignant melanoma in women. J Natl Cancer Inst (2003) 95(20):1530–1538.[Abstract/Free Full Text]
31. Veierød MB, Weiderpass E, Lund E, et al. Re: A prospective study of pigmentation, sun exposure, and risk of cutaneous malignant melanoma in women [letter]. J Natl Cancer Inst (2004) 96(4):337–338.[Free Full Text]

CiteULike    Connotea    Del.icio.us    What's this?

Related articles in Am. J. Epidemiol.:

Recall Bias in Melanoma Risk Factors and Measurement Error Effects: A Nested Case-Control Study Within the Norwegian Women and Cancer Study

Christine L. Parr, Anette Hjartåker, Petter Laake, Eiliv Lund, and Marit B. Veierød
Am. J. Epidemiol. 2009 169: 257-266. [Abstract] [FREE Full Text]  

This article has been cited by other articles:

				A. E. Cust, H. Schmid, J. A. Maskiell, J. Jetann, M. Ferguson, E. A. Holland, C. Agha-Hamilton, M. A. Jenkins, J. Kelly, R. F. Kefford, et al. Population-based, Case-Control-Family Design to Investigate Genetic and Environmental Influences on Melanoma Risk: Australian Melanoma Family Study Am. J. Epidemiol., December 15, 2009; 170(12): 1541 - 1554. [Abstract] [Full Text] [PDF]

				C. L. Parr, A. Hjartaker, P. Laake, E. Lund, and M. B. Veierod Parr et al. Respond to "Recall Bias in Melanoma" Am. J. Epidemiol., February 1, 2009; 169(3): 271 - 272. [Full Text] [PDF]

This Article Abstract FREE Full Text (PDF) All Versions of this Article: 169/3/267    most recent kwn362v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Related articles in Am. J. Epidemiol. Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Search for citing articles in: ISI Web of Science (2) Request Permissions Disclaimer Google Scholar Articles by Gefeller, O. Search for Related Content PubMed PubMed Citation Articles by Gefeller, O. Social Bookmarking          What's this?

Online ISSN 1476-6256 - Print ISSN 0002-9262

Copyright © 2010 Johns Hopkins Bloomberg School of Public Health

Oxford Journals Oxford University Press

• Site Map
• Privacy Policy
• Frequently Asked Questions

Other Oxford University Press sites: Oxford University Press Oxford Journals China Oxford Journals Japan American National Biography Booksellers' Information Service Children's Fiction and Poetry Children's Reference Corporate & Special Sales Dictionaries Dictionary of National Biography Digital Reference English Language Teaching Higher Education Textbooks Humanities International Education Unit Law Medicine Music Online Products Oxford English Dictionary Reference Rights and Permissions Science School Books Social Sciences Very Short Introductions World's Classics