Skip Navigation


American Journal of Epidemiology Advance Access originally published online on July 8, 2008
American Journal of Epidemiology 2008 168(4):374-383; doi:10.1093/aje/kwn156
This Article
Right arrow Full Text Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
168/4/374    most recent
kwn156v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Related articles in Am. J. Epidemiol.
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Ioannidis, J. P. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ioannidis, J. P. A.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

American Journal of Epidemiology © The Author 2008. Published by the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

PRACTICE OF EPIDEMIOLOGY

Effect of Formal Statistical Significance on the Credibility of Observational Associations

John P. A. Ioannidis1,2,3

1 Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece
2 Biomedical Research Institute, Foundation for Research and Technology-Hellas, Ioannina, Greece
3 Department of Medicine, Tufts University School of Medicine, Boston, MA

Correspondence to Dr. John P. A. Ioannidis, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece (e-mail: jioannid{at}cc.uoi.gr).

Received for publication May 22, 2007. Accepted for publication January 4, 2008.

The author evaluated the implications of nominal statistical significance for changing the credibility of null versus alternative hypotheses across a large number of observational associations for which formal statistical significance (p < 0.05) was claimed. Calculation of the Bayes factor (B) under different assumptions was performed on 272 observational associations published in 2004–2005 and a data set of 50 meta-analyses on gene-disease associations (752 studies) for which statistically significant associations had been claimed (p < 0.05). Depending on the formulation of the prior, statistically significant results offered less than strong support to the credibility (B > 0.10) for 54–77% of the 272 epidemiologic associations for diverse risk factors and 44–70% of the 50 associations from genetic meta-analyses. Sometimes nominally statistically significant results even decreased the credibility of the probed association in comparison with what was thought before the study was conducted. Five of six meta-analyses with less than substantial support (B > 0.032) lost their nominal statistical significance in a subsequent (more recent) meta-analysis, while this did not occur in any of seven meta-analyses with decisive support (B < 0.01). In these large data sets of observational associations, formal statistical significance alone failed to increase much the credibility of many postulated associations. Bayes factors may be used routinely to interpret "significant" associations.

Bayes theorem; empirical research; epidemiologic methods; meta-analysis; observation; statistics

Abbreviations: RR, relative risk

Editor's note: An invited commentary on this article appears on page 384, and the author's response appears on page 389.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?

Related articles in Am. J. Epidemiol.:

Invited Commentary: Evidence-based Evaluation of p Values and Bayes Factors
Hormuzd A. Katki
Am. J. Epidemiol. 2008 168: 384-388. [Abstract] [FREE Full Text]  

The Author Responds to "Evaluating p Values and Bayes Factors"
John P. A. Ioannidis
Am. J. Epidemiol. 2008 168: 389-390. [Extract] [FREE Full Text]  



This article has been cited by other articles:


Home page
 JAMAHome page
I. Tzoulaki, G. Liberopoulos, and J. P. A. Ioannidis
Assessment of Claims of Improved Prediction Beyond the Framingham Risk Score
JAMA, December 2, 2009; 302(21): 2345 - 2352.
[Abstract] [Full Text] [PDF]

Home page
 Sci Transl MedHome page
J. P. A. Ioannidis, E. Y. Loy, R. Poulton, and K. S. Chia
Researching Genetic Versus Nongenetic Determinants of Disease: A Comparison and Proposed Unification
Science Translational Medicine, November 18, 2009; 1(7): 7ps8 - 7ps8.
[Full Text] [PDF]

Home page
 Hum Mol GenetHome page
B. Rueda, J. Broen, C. Simeon, R. Hesselstrand, B. Diaz, H. Suarez, N. Ortego-Centeno, G. Riemekasten, V. Fonollosa, M.C. Vonk, et al.
The STAT4 gene influences the genetic predisposition to systemic sclerosis phenotype
Hum. Mol. Genet., June 1, 2009; 18(11): 2071 - 2077.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
J. P. A. Ioannidis
The Author Responds to "Evaluating p Values and Bayes Factors"
Am. J. Epidemiol., August 15, 2008; 168(4): 389 - 390.
[Full Text] [PDF]

Home page
 Am J EpidemiolHome page
H. A. Katki
Invited Commentary: Evidence-based Evaluation of p Values and Bayes Factors
Am. J. Epidemiol., August 15, 2008; 168(4): 384 - 388.
[Abstract] [Full Text] [PDF]


Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.