PRACTICE OF EPIDEMIOLOGY |
Interrater Reliability: Completing the Methods Description in Medical Records Review Studies
From the Department of Research, Olmsted Medical Center, Rochester, MN
Correspondence to Dr. Barbara P. Yawn, Department of Research, Olmsted Medical Center, 210 Ninth Street, SE, Rochester, MN 55904 (e-mail: yawnx002{at}umn.edu).
In medical records review studies, information on the interrater reliability (IRR) of the data is seldom reported. This study assesses the IRR of data collected for a complex medical records review study. Elements selected for determining IRR included "demographic" data that require copying explicit information (e.g., gender, birth date), "free-text" data that require identifying and copying (e.g., chief complaints and diagnoses), and data that require abstractor judgment in determining what to record (e.g., whether heart disease was considered). Rates of agreement were assessed by the greatest number of answers (one to all n) that were the same. The IRR scores improved over time. At 1 month, the reliability for demographic data elements was very good, for free-text data elements was good, but for data elements requiring abstractor judgment was unacceptable (only 3.4 of six answers agreed, on average). All assessments after 6 months showed very good to excellent IRR. This study demonstrates that IRR can be evaluated and summarized, providing important information to the study investigators and to the consumer for assessing the reliability of the data and therefore the validity of the study results and conclusions. IRR information should be required for all large medical records studies.
abstracting and indexing; data collection; epidemiologic methods; medical records; reproducibility of results; research design
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  L. Engel, C. Henderson, J. Fergenbaum, and A. Colantonio Medical Record Review Conduction Model for Improving Interrater Reliability of Abstracting Medical-Related Information Eval Health Prof, September 1, 2009; 32(3): 281 - 298. [Abstract] [PDF]
|
|
|
|
 |
|
 C. L. Kimberlin and A. G. Winterstein Validity and reliability of measurement instruments used in research Am. J. Health Syst. Pharm., December 1, 2008; 65(23): 2276 - 2284. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. L. Arms, H. Ortega, and S. Reid Chronological and Clinical Characteristics of Apnea Associated With Respiratory Syncytial Virus Infection: A Retrospective Case Series Clinical Pediatrics, November 1, 2008; 47(9): 953 - 958. [Abstract] [PDF]
|
|
|
|
 |
|
 B. P. Yawn, P. Saddier, P. C. Wollan, J. L. St. Sauver, M. J. Kurland, and L. S. Sy A Population-Based Study of the Incidence and Complication Rates of Herpes Zoster Before Zoster Vaccine Introduction Mayo Clin. Proc., November 1, 2007; 82(11): 1341 - 1349. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. L. Lash, M. P. Fox, S. S. Thwin, A. M. Geiger, D. S. M. Buist, F. Wei, T. S. Field, M. U. Yood, F. J. Frost, V. P. Quinn, et al. Using Probabilistic Corrections to Account for Abstractor Agreement in Medical Record Reviews Am. J. Epidemiol., June 15, 2007; 165(12): 1454 - 1461. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. W. Goldberg, J. A. Kreyenbuhl, D. R. Medoff, F. B. Dickerson, K. Wohlheiter, L. J. Fang, C. H. Brown, and L. B. Dixon Quality of Diabetes Care Among Adults With Serious Mental Illness Psychiatr Serv, April 1, 2007; 58(4): 536 - 543. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Sheikh RE: "INTERRATER RELIABILITY: COMPLETING THE METHODS DESCRIPTION IN MEDICAL RECORDS REVIEW STUDIES" Am. J. Epidemiol., November 1, 2005; 162(9): 919 - 919. [Full Text] [PDF]
|
|
|
|
|
|
B. P. Yawn and P. M. Wollan THE AUTHORS REPLY Am. J. Epidemiol., November 1, 2005; 162(9): 919 - 920. [Full Text] [PDF]
|
|