American Journal of Epidemiology Advance Access originally published online on April 23, 2007
American Journal of Epidemiology 2007 166(2):130-136; doi:10.1093/aje/kwm062
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
PRACTICE OF EPIDEMIOLOGY |
Validity of Self-Reported Endoscopies of the Large Bowel and Implications for Estimates of Colorectal Cancer Risk
1 Division of Clinical Epidemiology and Aging Research, German Cancer Research Center, Heidelberg, Germany
2 Division of Cancer Epidemiology, Unit of Genetic Epidemiology, German Cancer Research Center, Heidelberg, Germany
Correspondence to Dr. Hermann Brenner, Division of Clinical Epidemiology and Aging Research, German Cancer Research Center, Bergheimer Strasse 20, D-69115 Heidelberg, Germany (e-mail: h.brenner{at}dkfz-heidelberg.de).
Received for publication October 17, 2006. Accepted for publication January 22, 2007.
| ABSTRACT |
|---|
|
|
|---|
Large-bowel endoscopy with removal of polyps strongly reduces colorectal cancer risk. In epidemiologic studies, ascertainment of large-bowel endoscopies often relies on self-reports and might be prone to imperfect recall. In 20032004, the authors assessed the validity of self-reported colorectal endoscopies in a population-based case-control study including 540 cases and 614 controls from southwest Germany and calculated odds ratios of colorectal cancer risk according to self-reports and medical records. They sought to obtain all medical records for the last self-reported endoscopy and for a subsample of 100 subjects reporting no previous endoscopy. In total, 377 of 483 records could be obtained (78%). Sensitivity of self-reports was 100%, and specificity ranged from 93% to 98% among subgroups defined by age, gender, education, family history of colorectal cancer, and case-control status. The odds ratios for colorectal cancer risk after previous colonoscopy were 0.31 (95% confidence interval: 0.21, 0.45) using self-reports and 0.31 (95% confidence interval: 0.20, 0.47) using medical records. However, agreement between self-reports and medical records was poor regarding type of endoscopy (colonoscopy, sigmoidoscopy, or rectoscopy; kappa = 0.22), moderate concerning polypectomy (kappa = 0.58), and reasonable for year of examination (kappa = 0.70). Self-reports of previous colorectal endoscopies agreed well with medical records, but validation appears to be essential with respect to details of the examination.
colonoscopy; colorectal neoplasms; endoscopy; mass screening; reproducibility of results; risk; validation studies
| INTRODUCTION |
|---|
|
|
|---|
A history of large-bowel screening endoscopy is among the strongest protective factors against colorectal cancer (18). Its ascertainment in previous epidemiologic studies often relied on self-reports (e.g., Slattery et al. (6), Brenner et al. (7), Giovannucci et al. (9), Gann et al. (10), and Chan et al. (11)) and may thus be prone to imperfect recall, despite the invasive character of the examinations. Potential imperfect recall might threaten the validity of studies on the impact of screening endoscopy on colorectal cancer risk. The few studies investigating the validity of self-reported colorectal endoscopies based on medical records as a standard to authenticate self-reports were conducted mostly among members of health maintenance organizations in the United States (1215). To date, we know of only one study that has validated self-reports in a population-based setting with relatives of patients of a colon cancer registry in Canada (16).
We assessed the validity of self-reported previous endoscopy of the large bowel by comparing self-reports with medical records among participants of a large population-based case-control study. In addition, we compared odds ratios for the association of previous colonoscopy with the risk of colorectal cancer when relying on either self-reports or medical records only.
| MATERIALS AND METHODS |
|---|
|
|
|---|
Study population
We obtained detailed information about previous colorectal endoscopies from participants of the DACHS study. A more extensive description of this study has been published elsewhere (8). In brief, the DACHS study, a population-based case-control study in the Rhine-Neckar region in the southwest of Germany, was initiated to assess the potential of screening endoscopies for the prevention of colorectal cancer. Participants were included if they were at least 30 years of age, German speaking, and physically and mentally able to take part in a standardized questionnaire-based interview of about 1 hour. Patients with incident colorectal cancer were informed about the study by their treating physicians, mostly during their hospital stay a few days after surgery. After the physicians received their informed consent, the study center was notified about the participants. Controls were randomly drawn from population registers and were (frequency) matched to cases by age, gender, and county of residence before they were contacted by the study center through mail and follow-up calls. Between January 2003 and June 2004, 540 patients with incident colorectal cancer and 614 controls without colorectal cancer were recruited.
Assessment of endoscopies of the large bowel
Data on previous endoscopies of the large bowel as well as on potential risk factors for colorectal cancer were obtained through personal, standardized interviews, which were conducted by a small team of trained interviewers and lasted about 1 hour. All questions regarding endoscopies of the large bowel referred to the time before the onset of symptoms (patients) or the time before the interview (controls).
Participants were asked whether they had ever had an endoscopy of the large bowel and, if so, when the last one was performed, how often in total, and how often during the past 5 years. For each of up to four endoscopies per participant, we asked for the reason, the endoscopist or hospital where the examination occurred, the procedure applied (colonoscopy, sigmoidoscopy, or rectoscopy), and whether polyps had been removed. If the participant was uncertain or did not know the procedure by name, the interviewer explained the typical preparations, circumstances, and settings.
Requests for medical records of previous endoscopies
We requested medical records for the last self-reported endoscopy of the large bowel from the subjects' physicians or hospitals according to a comprehensive monitoring scheme. In a first step, as many as two physicians who could possibly have a medical record of the endoscopy (endoscopist, general practitioner, or hospital) were contacted by letter and, if there was no reply after 4 weeks, were followed up by phone. We first contacted the general practitioner; if no or not all records could be obtained, we next contacted the endoscopists reported by the participant or the general practitioner. If there was no information about a second physician, we asked the participant again where we might obtain a matching medical record for the self-reported colorectal endoscopy. If we obtained additional information from the participant, we also inquired about a medical record at that location.
A subsample of 100 subjects was randomly drawn from the 768 participants with no self-report of a previous large-bowel endoscopy. We requested that the subjects' general practitioner either confirm or invalidate the negative self-report by the participant or to state that available information was insufficient to exclude a previous colorectal endoscopy of the patient. In the latter instance, we asked about a second physician who might possibly have more information and we contacted that physician.
Comparing self-reported endoscopies with medical records
A positive self-report was considered true when confirmed by a medical record or the physician's report, and false if the consulted physician could reliably state that no such examination was obtained by the patient or if the last colorectal endoscopy was in fact a misreport of another examination (e.g., gastroscopy). A negative self-report was considered true if the subject's general practitioner was able to confirm that no previous endoscopy of the large bowel had been performed, and false if the general practitioner or a medical record documented a previous colorectal endoscopy.
Statistical analysis
This validation study was restricted to all participants with a positive self-report and a random subsample of 100 participants with a negative self-report of a previous endoscopy of the large bowel. We described characteristics of and differences (chi-square test) between subjects with a self-report and those without a reported colorectal endoscopy, and we calculated statistics for the validity of self-reports and their agreement with medical records (positive and negative predictive values, sensitivity, specificity, concordance, kappa statistic). In this context, sensitivity applied to the proportion of those with a colorectal endoscopy who also reported this procedure during the interview. Likewise, specificity applied to the proportion of subjects who did not have an endoscopy of the large bowel and who correctly reported so in the interview. Concordance and the kappa statistic measure the overall and chance-corrected agreement of self-reports and medical records, respectively. In the analyses of measures of validity and agreement, we accounted for the different sampling fractions of subjects with and without self-reported endoscopy by performing weighting of subjects (with weights equal to the reciprocal of the sampling fraction within each group).
Among subjects who self-reported endoscopy of the large bowel, we measured agreement of self-reports and medical records for the type of endoscopy (colonoscopy, sigmoidoscopy, or rectoscopy) for detection and removal of polyps at last endoscopy (yes/no) and for the year of last endoscopy (
2, 34, 59, 1019,
20 years agocategories as used in the analyses of risk reduction according to time since colonoscopy (8)).
We further estimated the association of previous colorectal endoscopy according to self-reports or medical records with colorectal cancer in the entire sample by multiple logistic regression. Odds ratios were adjusted for the matching variables age, gender, and county of residence and for the following other known or suspected risk or protective factors for colorectal cancer: body mass index, educational level (eight categories according to the German Association of Epidemiology (17)), first-degree family history of colorectal cancer (yes/no, unknown), participation in a former health check-up (ever/never), lifetime pack-years of active smoking (never active/<10/1019/2029/
30), ever regular use of nonsteroidal antiinflammatory drugs two or more times per week over 1 or more years (yes/no), and ever regular use of hormone replacement therapy (yes/no).
All analyses were carried out with the statistical software package SAS 9.1 (SAS Institute, Inc., Cary, North Carolina).
| RESULTS |
|---|
|
|
|---|
Of the 1,154 participants, 1,152 provided information about a previous endoscopy of the large bowel (figure 1). Among these participants, 384 reported ever having had such an examination, and, for 293 participants, we were able to obtain medical records of the last colorectal endoscopy (77 percent of requests). Of the 768 subjects who reported no previous endoscopy of the large bowel, we drew a random sample of 100 subjects and requested their general practitioner to confirm or invalidate the report of the participant whenever possible; we received 84 records. The most frequent reasons why medical records could not be obtained were because they had been destroyed (these documents need to be stored no longer than 10 years in Germany) or they were not available (anymore) from the physicians or hospitals contacted.
|
We found no major differences in the characteristics of subjects with positive and negative reports of a previous endoscopy of the large bowel (table 1), except for a much higher proportion of cases among those with no report of a previous endoscopy (p < 0.01). Most endoscopies reported were conducted within the past 5 years (70 percent), and 17 percent and 12 percent were conducted 610 years or more than 10 years ago, respectively (1 percent of the reports were missing). The last reported colorectal endoscopy was a colonoscopy, sigmoidoscopy, and rectoscopy for 87 percent, 2 percent, and 7 percent of cases, respectively (4 percent unknown or missing).
|
Self-reports of previous large-bowel endoscopy could be confirmed for 266 of 293 cases (positive predictive value, 91 percent) (figure 1, table 1). A previous self-reported endoscopy was wrongly reported by 27 subjects, who mostly erroneously reported gastroscopy or gave an incorrect account for unknown reasons. Absence of a previous large-bowel endoscopy could be confirmed for all of 84 cases (negative predictive value, 100 percent). Weighted analyses taking into account the different sampling fractions in both groups (293 of 384 positive self-reports, 84 of 768 negative self-reports) yielded estimates of overall sensitivity and specificity of 100 percent and 96 percent, respectively, and of overall concordance and kappa of 97 percent and 0.93, respectively. Specificity was highest for subjects with a high educational level (98 percent), and it was higher among cases than among controls.
Among participants with a self-reported previous endoscopy of the large bowel, we compared the self-reported endoscopic procedure with the types of procedures indicated in the medical records (table 2). Overall concordance was 85 percent. However, this high level of concordance was mostly due to the high proportion of colonoscopies performed and reported by participants (kappa = 0.22). Regarding self-reports of polyp removal at last endoscopy, concordance was 84 percent with moderate chance-corrected agreement (kappa = 0.58) (table 3). In most instances, subjects reporting no polyp removal also had no account of polyp removal in the medical records (95 percent). However, only 39 of 67 subjects (58 percent) reporting polypectomy at last endoscopy also had an account of polypectomy in the medical records. The reported time of last endoscopy agreed with the time of endoscopy in the medical records for 79 percent of cases (kappa = 0.70) (table 4). Patients who had their last endoscopy up to 4 years ago reported the year more reliably (85 percent) than those whose last examination was more than 4 years ago (69 percent).
|
|
|
We analyzed the effect of screening colonoscopies and reduction of colorectal cancer risk according to self-reports of colonoscopy and according to medical records of colonoscopy, respectively. In this analysis, 293 subjects with validated reports of a previous endoscopy of the large bowel and 768 subjects with validated (n = 84) or nonvalidated (n = 684) reports of no previous endoscopy were included (figure 1). Of these 503 cases and 558 controls, we then excluded 101 cases who were detected by screening and two with a previous endoscopy less than 1 year ago. The former exclusion is required for valid assessment of the impact of screening on colorectal cancer risk (18). However, self-reports of previous endoscopies conducted for screening and other purposes were considered among both cases and controls. The latter exclusion was made to avoid considering endoscopies that might have been part of the diagnostic process. We also excluded 36 controls who had had endoscopies less than 1 year ago to ensure comparability of cases and controls. To focus on reduction in risk of colorectal cancer achieved by colonoscopy, we excluded another four cases and 10 controls with rectoscopy, sigmoidoscopy, or no specification of endoscopic procedure in both self-reports and medical records. Of the remaining 396 cases and 512 controls, four cases and four controls were not considered because only hyperplastic polyps were removed according to medical records, leaving 392 cases and 508 controls for this part of the analyses.
We found no difference in the estimates of overall risk reduction after previous colonoscopy ascertained by self-report or medical records (adjusted odds ratios = 0.31 and 0.31, respectively; table 5). Likewise, no differences in risk estimates for subjects with a previous negative colonoscopy (no polyps detected and removed) were found (adjusted odds ratios = 0.25 and 0.25, respectively). There appeared to be a difference in odds ratios for previous colonoscopy with polypectomy, but confidence intervals were broader and widely overlapping.
|
| DISCUSSION |
|---|
|
|
|---|
In this population-based study, we assessed the validity of self-reported history of colorectal endoscopy. Using medical records or physicians' reports to validate self-reports, we found very high overall sensitivity and sensitivity of 100 percent and 96 percent, respectively. Overreporting of previous endoscopy of the large bowel ranged from 2 percent to 7 percent among the different subgroups only. Although a previous colonoscopy was well reported, distinction between colonoscopy (by far the most frequent endoscopic procedure applied in this population), sigmoidoscopy, and rectoscopy beyond chance was poor (kappa = 0.22). Popularity of the term "colonoscopy" in Germany rather than true ability of patients to distinguish between endoscopies may be the reason for the high percentage of overall agreement (85 percent), a suggestion supported by the poor agreement for sigmoidoscopy and rectoscopy. Previous polypectomy could be confirmed for only 58 percent of cases. Agreement of reported year of last endoscopy with medical records was reasonable (79 percent), but reports were less reliable if the examination was more than 4 years ago. Odds ratios for the association between previous colonoscopy and colorectal cancer did not materially change when information on the endoscopy was drawn from medical records rather than from self-reports.
To our knowledge, only one other study audited 79 self-reports of sigmoidoscopy and 169 self-reports of colonoscopy in a population-based setting with relatives of registered colorectal cancer patients from Canada, and that study found respective sensitivities and specificities that were high for colonoscopy (95 percent and 92 percent) and somewhat lower for sigmoidoscopy (79 percent and 80 percent) (16). The authors noted that their study population was more health conscious because of the family history of colorectal cancer and that the results may not be applicable to the general population. The small fraction of those with a previous rectoscopy and sigmoidoscopy included, sensitivity and specificity were comparably high in our study (100 percent sensitivity, 96 percent specificity).
The few other pertinent studies were carried out within single or multiple health maintenance organizations in the United States, where medical records are readily available (1215). For colonoscopy and sigmoidoscopy combined, one previous study found sensitivity of 96 percent and specificity of 92 percent within a single health maintenance organization (15). In another study that examined the validity of self-reported sigmoidoscopy or colonoscopy in Black men, White men (including other), and women for each of the three health maintenance organizations studied, sensitivities and specificities ranged mostly from 85 percent to 96 percent and 71 percent to 88 percent, respectively (14). In these studies, it is unlikely that colorectal endoscopies were performed outside the prepaid health plan (15). However, validity could be assessed for the last 25 years only, and health plan members may not be representative of the general population or populations with other health insurance systems. One other study examined test-retest reliability of colorectal cancer tests in telephone interviews and found reasonable concordance of 85 percent for ever having had an endoscopy (kappa = 0.69) (19).
Baier et al. (14) examined validity of self-reported sigmoidoscopy and colonoscopy by age in subjects aged 4075 years, suggesting a decline with increasing age (not statistically significant). Age had no effect on the validity of self-reports in the present study, which had no upper age limit.
The high levels of sensitivity and specificity and the similarity of odds ratios for colorectal cancer among subjects with previous colonoscopy (overall or those with a negative colonoscopy only) are reassuring and suggest that the association may be validly estimated in studies relying exclusively on self-reports (6, 7, 20). For such studies, the small benefits of validation have to be weighed against potentially substantial costs and efforts of validation. In particular, validation of a self-reported negative history of colorectal endoscopy may often not be worthwhile. On the other hand, validation appears to be essential if more specific information, such as type of endoscopy or findings of polyps, are of interest. Validity of self-reports of these features may be enhanced to some extent if a comprehensive explanation of the special circumstances and settings is provided to the participant before asking about colorectal endoscopies (and not only in case of uncertainty, as in our study). Nevertheless, it appears unlikely that acceptable validity of self-reports regarding details of endoscopic examination can be achieved even with such efforts.
The strengths of the present study include the population-based design and the comprehensive assessment of previous endoscopies of the large bowel during the interview, as well as the extensive validation scheme. We requested medical records from the patients' physicians or hospitals, and we had no limit regarding time period since the last examination.
There are also some limitations of this study to consider. This validation study was conducted within a case-control study of colorectal cancer, and the proportion of colorectal cancer patients (33 percent) reporting an endoscopy was much higher than in the general population. Overreporting of a previous colorectal endoscopy was less pronounced among persons with colorectal cancer (2 percent) than among persons without colorectal cancer (7 percent), so overall specificity might be somewhat closer to 93 percent if this study were to be conducted in the normal population with a lower prevalence of colorectal cancer. As in previous studies, we audited self-reports by considering medical records as standard, although previous endoscopies might sometimes still have been missed because of a lack of records, which would have led to underestimation of the specificity and overestimation of sensitivity. In addition, if physicians could not confirm or invalidate a negative self-report because of insufficient information about the patient (15 of 100), a potential previous endoscopy of the large bowel could not be ruled out entirely. If some of these subjects had a previous colorectal endoscopy, sensitivity and negative predictive value would be slightly overestimated. The controls in the present study might be somewhat more health conscious and more likely than the general population to have had a screening examination of the large bowel. Only a few participants reported sigmoidoscopy or rectoscopy, so the results of this study can best be compared with other validation studies of self-reported colonoscopies.
In conclusion, self-reports of previous endoscopies of the large bowel agreed well with medical records in this large population-based validation study and might not require further validation, if no further details about the procedure are of interest. However, validation appears to be essential whenever detailed information about the type of examination and endoscopic findings is required for analyses.
| ACKNOWLEDGMENTS |
|---|
The authors thank the study interviewers who collected the data, and they are grateful to the cooperating physicians and hospitals in Germany for their support and assistance in recruiting patients for this study: Chirurgische Universitätsklinik Heidelberg, Klinik am Gesundbrunnen Heilbronn, St. Vincentiuskrankenhaus Speyer, St. Josefskrankenhaus Heidelberg, Chirurgische Universitätsklinik Mannheim, Diakonissenkrankenhaus Speyer, Krankenhaus Salem Heidelberg, Kreiskrankenhaus Schwetzingen, St. Marien und St. Annastiftkrankenhaus Ludwigshafen, Klinikum Ludwigshafen, Stadtklinik Frankenthal, Diakoniekrankenhaus Mannheim, Kreiskrankenhaus Sinsheim, Klinikum am Plattenwald Bad Friedrichshall, Kreiskrankenhaus Weinheim, Kreiskrankenhaus Eberbach, Kreiskrankenhaus Buchen, Kreiskrankenhaus Mosbach, Enddarmzentrum Mannheim, and Kreiskrankenhaus Brackenheim. The authors also thank U. Handte-Daub, C. El Idrissi-Lamghari, S. Toth, and B. Collins for technical assistance over the course of this study.
Conflict of interest: none declared.
| References |
|---|
|
|
|---|
- Selby JV, Friedman GD, Quesenberry CP Jr, et al. A case-control study of screening sigmoidoscopy and mortality from colorectal cancer. N Engl J Med (1992) 326:6537.[Abstract]
- Newcomb PA, Norfleet RG, Storer BE, et al. Screening sigmoidoscopy and colorectal cancer mortality. J Natl Cancer Inst (1992) 84:15725.
[Abstract/Free Full Text] - Winawer SJ, Zauber AG, Ho MN, et al. Prevention of colorectal cancer by colonoscopic polypectomy. The National Polyp Study Workgroup. N Engl J Med (1993) 329:197781.
[Abstract/Free Full Text] - Muller AD, Sonnenberg A. Protection by endoscopy against death from colorectal cancer. A case-control study among veterans. Arch Intern Med (1995) 155:17418.[Abstract]
- Muller AD, Sonnenberg A. Prevention of colorectal cancer by flexible endoscopy and polypectomy. A case-control study of 32,702 veterans. Ann Intern Med (1995) 123:90410.
[Abstract/Free Full Text] - Slattery ML, Edwards SL, Ma KN, et al. Colon cancer screening, lifestyle, and risk of colon cancer. Cancer Causes Control (2000) 11:55563.[CrossRef][ISI][Medline]
- Brenner H, Arndt V, Sturmer T, et al. Long-lasting reduction of risk of colorectal cancer following screening endoscopy. Br J Cancer (2001) 85:9726.[CrossRef][ISI][Medline]
- Brenner H, Chang-Claude J, Seiler CM, et al. Does a negative screening colonoscopy ever need to be repeated? Gut (2006) 55:114550.
[Abstract/Free Full Text] - Giovannucci E, Rimm EB, Stampfer MJ, et al. Aspirin use and the risk for colorectal cancer and adenoma in male health professionals. Ann Intern Med (1994) 121:2416.
[Abstract/Free Full Text] - Gann PH, Manson JE, Glynn RJ, et al. Low-dose aspirin and incidence of colorectal tumors in a randomized trial. J Natl Cancer Inst (1993) 85:12204.
[Abstract/Free Full Text] - Chan AT, Giovannucci EL, Meyerhardt JA, et al. Long-term use of aspirin and nonsteroidal anti-inflammatory drugs and risk of colorectal cancer. JAMA (2005) 294:91423.
[Abstract/Free Full Text] - Gordon NP, Hiatt RA, Lampert DI. Concordance of self-reported data and medical record audit for six cancer screening procedures. J Natl Cancer Inst (1993) 85:56670.
[Abstract/Free Full Text] - Hiatt RA, Perez-Stable EJ, Quesenberry C Jr, et al. Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California. Prev Med (1995) 24:27885.[CrossRef][ISI][Medline]
- Baier M, Calonge N, Cutter G, et al. Validity of self-reported colorectal cancer screening behavior. Cancer Epidemiol Biomarkers Prev (2000) 9:22932.
[Abstract/Free Full Text] - Hall HI, Van Den Eeden SK, Tolsma DD, et al. Testing for prostate and colorectal cancer: comparison of self-report and medical record audit. Prev Med (2004) 39:2735.[CrossRef][ISI][Medline]
- Madlensky L, McLaughlin J, Goel V. A comparison of self-reported colorectal cancer screening with medical records. Cancer Epidemiol Biomarkers Prev (2003) 12:6569.
[Abstract/Free Full Text] - Ahrens W, Bellach BM, Jöckel KH. Assessment of sociodemographic characteristics in epidemiology. (In German) (1998) Munich, Germany: MMV Medizin Verlag.
- Weiss NS. Case-control studies of the efficacy of screening tests designed to prevent the incidence of cancer. Am J Epidemiol (1999) 149:14.
[Free Full Text] - Bradbury BD, Brooks DR, Brawarsky P, et al. Test-retest reliability of colorectal testing questions on the Massachusetts Behavioral Risk Factor Surveillance System (BRFSS). Prev Med (2005) 41:30311.[CrossRef][ISI][Medline]
- Newcomb PA, Storer BE, Morimoto LM, et al. Long-term efficacy of sigmoidoscopy in the reduction of colorectal cancer incidence. J Natl Cancer Inst (2003) 95:6225.
[Abstract/Free Full Text]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
