Skip Navigation


American Journal of Epidemiology Advance Access originally published online on January 12, 2006
American Journal of Epidemiology 2006 163(4):300-309; doi:10.1093/aje/kwj046
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
163/4/300    most recent
kwj046v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (31)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Trikalinos, T. A.
Right arrow Articles by Ioannidis, J. P. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Trikalinos, T. A.
Right arrow Articles by Ioannidis, J. P. A.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

American Journal of Epidemiology Copyright © 2006 by the Johns Hopkins Bloomberg School of Public Health All rights reserved; printed in U.S.A.

Meta-Analysis

Impact of Violations and Deviations in Hardy-Weinberg Equilibrium on Postulated Gene-Disease Associations

Thomas A. Trikalinos1,2, Georgia Salanti3,4, Muin J. Khoury5 and John P. A. Ioannidis1,2,6

1 Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece
2 Department of Medicine, Tufts University School of Medicine, Boston, MA
3 Biostatistics Unit, Medical Research Council, Cambridge, United Kingdom
4 Cambridge Genetics Knowledge Park, Cambridge, United Kingdom
5 Office of Genomics and Disease Prevention, Centers for Disease Control and Prevention, Atlanta, GA
6 Biomedical Research Institute, Foundation for Research and Technology-Hellas, Ioannina, Greece

Correspondence to Dr. John P. A. Ioannidis, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece (e-mail: jioannid{at}cc.uoi.gr).

Received for publication April 20, 2005. Accepted for publication September 28, 2005.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
The authors evaluated whether statistically significant violations of Hardy-Weinberg equilibrium (HWE) or the magnitude of deviations from HWE may contribute to the problem of replicating postulated gene-disease associations across different studies. Forty-two gene-disease associations assessed in meta-analyses of 591 studies were examined. Studies with disease-free controls in which HWE was violated gave significantly different results from HWE-conforming studies in five instances. Exclusion of the former studies resulted in loss of statistical significance of the overall meta-analysis in three instances and more than a 10% change in the summary odds ratio in six. Exclusion of HWE-violating studies changed the formal significance of the estimated between-study heterogeneity in three instances. After adjustment for the magnitude of the deviation from HWE for the controls, formal significance was lost in another three instances. Studies adjusted for the magnitude of deviation from HWE tended to become more heterogeneous among themselves, and, for seven gene-disease associations, between-study heterogeneity became significant, while it was not so in the unadjusted analyses. Gene-disease association studies and meta-analyses thereof should routinely scrutinize the potential impact of HWE violations as well as nonsignificant deviations from the exact frequencies expected under HWE. Postulated genetic associations with modest-sized odds ratios and borderline statistical significance may not be robust in such sensitivity analyses.

association; bias (epidemiology); genes; genetics; meta-analysis

Abbreviations: HWE, Hardy-Weinberg equilibrium; ID, (operational) identification number


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Genetic associations for complex diseases are important to establish because they are expected to cumulatively account for a substantial proportion of susceptibility to many diseases of considerable public health impact (1Go, 2Go). However, concerns are increasing about the lack of replication of proposed gene-disease associations (3Go–5Go). Various reasons have been proposed for this phenomenon, including, among others, exaggerated early results (3Go), publication bias and time-lag bias against "negative" studies (3Go, 6Go), differences between small and larger studies (7Go), and population stratification and "racial/ethnic" heterogeneity (8Go). Empirical evidence has been accumulating on the relative role of some of these potential problems (3Go, 4Go, 7Go–9Go).

An additional major theoretical concern is that associations may be spurious if the distribution of genotypes in the healthy control groups in genetic case-control studies deviates from Hardy-Weinberg equilibrium (HWE). HWE depends on a series of assumptions about the tested population, including, for example, no new mutation, no selection, and random mating (10Go). Departures from HWE, if not due to chance or violation of these assumptions, therefore may point to genotyping error or other biases (11Go, 12Go). Empirical evidence suggests that in about 10 percent of case-control studies in the field, the distributions of genotypes in the healthy control group violate HWE; that is, they show formally statistically significant deviations from the HWE-expected frequencies (12Go–14Go). Nevertheless, investigators uncommonly report appropriately on testing and departures from HWE, even in genetics journals (14Go). Moreover, given the fact that most studies in the field have relatively small sample sizes, the power of available statistical tests to detect HWE violation is limited (14Go).

Despite these concerns, there is no known large-scale empirical evidence on whether HWE deviations may cause serious bias when estimating the magnitude of genetic associations in gene-disease association studies. Here, we examine whether studies that violate HWE give systematically different genetic effect sizes compared with studies in which HWE is not formally violated, and whether between-study heterogeneity may be explained in part by HWE violations. We also evaluate whether deviations from the exact frequencies expected under HWE, even if not formally statistically significant, may still affect the postulated genetic associations.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 APPENDIX
 References
 
Database
We used a database of 134 meta-analyses of binary disease outcomes with genetic markers not belonging to the human leukocyte antigen (HLA) system (last update, May 2004); details on the selection criteria, the search algorithms, and data extraction have been described previously (3Go, 7Go, 8Go). In brief, these meta-analyses 1) contained at least three studies published over two different calendar years and 2) pertained to binary outcomes. A comprehensive algorithm was used to select the main genetic contrast (3Go). Study-level numerical data were extracted either from the meta-analysis publication or after communication with the authors of each meta-analysis. Separate case-control comparisons of different "racial/ethnic"–descent populations described in the same report were treated as separate studies (8Go).

All these meta-analyses were screened for eligibility for the current project. Only those of biallelic markers were retained (15Go). Meta-analyses in which controls were not disease free and those in which the genotype frequencies in the controls were not available per study were also excluded because HWE testing was not possible. Relying on investigators' claims regarding HWE testing is known to be unreliable (14Go).

Information on genotype distribution in the controls included in each constituent study was recorded from each eligible meta-analysis. Data were extracted by two investigators. Discrepancies were discussed and were resolved in a consensus meeting.

HWE testing
In each constituent study, we tested for HWE by using an asymptotic {chi}2 test or an exact test (16Go) among the disease-free controls. The exact test was preferred when the expected count under the Hardy-Weinberg law was less than five for any of the three genotypes. For consistency, in this paper we use the terms "violations" to describe departures from HWE that reach formal statistical significance and "deviations" to describe any departure from the exact frequencies expected under HWE, regardless of whether the departure is formally statistically significant.

Meta-analyses: summary effects and heterogeneity testing
The odds ratio was used as the metric of choice. For each meta-analysis, we calculated the summary odds ratio by including all available studies, and then we calculated it separately for studies with statistically significant deviation from HWE and for HWE-conforming studies. Summary estimates were obtained with random-effects (DerSimonian and Laird) models (17Go), which allow for between-study variability and incorporate it in the calculations. Fixed-effects syntheses were not used because variability in the study results on the same association is very common (3Go, 7Go). We tested for between-study heterogeneity with Cochran's {chi}2-based Q statistic (considered significant for p < 0.10) (18Go), and we estimated its extent with the I2 statistic (19Go). I2 represents the percentage of the observed between-study variability due to heterogeneity rather than to chance. It ranges between 0 percent and 100 percent; values above 75 percent imply very large heterogeneity (19Go).

Impact of HWE-violating studies on summary effects and heterogeneity estimates
We evaluated whether the statistical significance status (at the 0.05 significance level) of each meta-analysis changed after HWE-violating studies were excluded. We also used a standardized z score to assess whether findings from HWE-violating studies differed from those of the HWE-conforming studies beyond chance (20Go). The z score is calculated by dividing the difference of the natural logarithms of the odds ratios in HWE-violating and in HWE-conforming studies by the standard deviation of this difference.

We also examined whether HWE-violating studies were responsible for some of the observed heterogeneity in each meta-analysis. The extent of between-study heterogeneity, as conveyed by the I2 and Q statistics, was compared before and after excluding HWE-violating studies.

Impact of correction for HWE deviations on summary effects and heterogeneity estimates
Finally, we assessed whether the magnitude, statistical significance, and between-study heterogeneity of the genetic association in each meta-analysis depended on the magnitude of deviations from the Hardy-Weinberg law, regardless of whether these deviations corresponded to formally statistically significant violations. The odds ratio and the variances of the individual studies were corrected to account for departures from HWE, as previously suggested (21Go, 22Go), and then each meta-analysis was performed again.

For allele-based contrasts, the variance of each study's effect size was adjusted by a factor of 1 + F (14Go, 21Go), where F is the inbreeding coefficient for the combined cases and controls. The inbreeding coefficient is calculated by Formula where PAA and Paa are the proportions of the homozygotes and pA and pa are the proportions of the corresponding alleles. The coefficient takes values from 1 to –1 depending on the extent of excess or deficit of homozygotes compared with the proportions expected under the Hardy-Weinberg law (15Go, 16Go).

For genotype-based contrasts, every study's odds ratio was corrected by using the HWE-predicted genotype counts in the control instead of the observed counts (22Go). The variances of the natural logarithms of the adjusted odds ratios were corrected as previously suggested (refer to the Appendix) (22Go). Corrected variances are usually smaller than those based on the observed genotype frequencies (22Go).

Software
Analyses were conducted with Intercooled Stata 8.2 software (Stata Corporation, College Station, Texas) by using the genhwi and metan modules, and R 1.9.1 (R Foundation for Statistical Computing, Vienna, Austria) by using the gap software package. In this paper, all p values are two tailed.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 APPENDIX
 References
 
Database
Forty-two meta-analyses with 591 studies met the inclusion criteria (table 1) (23Go–52Go), and 23 of these meta-analyses showed statistically significant gene-disease associations. Ten studies were excluded from the analyses (five part of a meta-analysis of 34 studies, and five part of a meta-analysis of 23 studies) because HWE could not be assessed (separate genotype counts were not available). Overall, control genotype frequencies in 60 studies (table 1) (60/591 = 10 percent, or even 60/581 = 10 percent, since HWE could not be assessed for 10 studies) from 27 meta-analyses violated the Hardy-Weinberg law. Fifteen meta-analyses did not include any such studies.


View this table:
[in this window]
[in a new window]
  		TABLE 1. Characteristics of the meta-analyses included in the present investigation*

 
Impact of HWE-violating studies
In 24 of 27 meta-analyses with at least one HWE-violating study, the statistical significance of the summary odds ratio remained the same before and after excluding the HWE-violating studies (figure 1). Nevertheless, the associations between the UCHL1 S18Y polymorphism and Parkinson's disease (11 studies, (operational) identification number (ID) 17 in table 1), between the MTHFR C677T polymorphism and venous thromboembolism (30 studies, ID 37), and between the KIR6.2BIR E23K polymorphism and type II diabetes (five studies, ID 41) were no longer formally significant after we excluded the HWE-violating studies. In these three instances, the summary p values before exclusion of the HWE-violating studies ranged between 0.007 and 0.025.


Figure 1
View larger version (28K):
[in this window]
[in a new window]
  		FIGURE 1. Meta-analyses that included or did not include studies violating Hardy-Weinberg equilibrium (HWE). Shown are 27 that included at least one association study in which departure from the Hardy-Weinberg law was statistically significant for the controls. Each meta-analysis is preceded by the (operational) identification number (ID) given in table 1, the first author of the study and the year of publication (e.g., '98 = 1998), and the reference number. For each meta-analysis, summary odds ratios and their 95% confidence intervals (CIs) are depicted. *, meta-analyses in which formal statistical significance was lost after HWE-violating studies were excluded.

 
In most meta-analyses, the absolute magnitude of the summary odds ratio was not substantially affected when HWE-violating studies were excluded from the calculations, but some exceptions were noted. The summary odds ratio changed by slightly over 10 percent in four meta-analyses and by 29 percent and 37 percent in two small meta-analyses with four and eight studies, respectively. In all other meta-analyses, the corresponding changes were much smaller (figure 1).

Studies showing significant departure from the Hardy-Weinberg law for the controls suggested either stronger (n = 13) or weaker (n = 14) estimates of association compared with the other studies (figure 1). In five instances, these discrepancies were beyond chance: HWE-violating studies showed no association between the ITGB3 L33P polymorphism and coronary artery disease or between the same marker and fractures (ID 6 and ID 7, respectively, in table 1), while other studies did. Conversely, HWE-violating studies claimed that the CYP17 MspI (ID 21) marker and the SRD5A2 A49T polymorphism (ID 23) conferred more than a twofold increased risk of prostate cancer, while the remaining studies showed no association. Similarly, a HWE-violating study showed an extreme association between the PON1 Q192R polymorphism and coronary stenosis (ID 31), while the effect was modest in the other studies (figure 1).

After we excluded HWE-violating studies, between-study heterogeneity lost its statistical significance for the association between the ITGB3 L33P polymorphism and fractures (ID 7 in table 1) and for the association between the MPO G-463A polymorphism and lung cancer (ID 40). Conversely, in the case of the MTHFR C677T polymorphism and neural tube defects (ID 3), the meta-analysis became significantly heterogeneous after we excluded the HWE-violating studies from our calculations. For the remaining 24 associations, there was no change in the formal significance status of heterogeneity. I2 did not consistently decrease or increase after exclusion of the HWE-violating studies (15 vs. nine instances, respectively, p = 0.18, adjusted for the three ties). Nevertheless, I2 decreased by more than 10 percent in six meta-analyses, whereas it did not increase by more than 10 percent in any instance (6 vs. 0, p = 0.014) (figure 2).


Figure 2
View larger version (18K):
[in this window]
[in a new window]
  		FIGURE 2. Extent of heterogeneity before and after exclusion of studies that violated Hardy-Weinberg equilibrium (HWE). I2 is shown before and after exclusion of studies in which the Hardy-Weinberg law was violated for the controls. The middle diagonal line is the line of equality. For studies plotted inside the area defined by the parallel lines above and below the line of equality, the change in I2 was less than 10%. The dashed reference lines correspond to I2 = 75%, the boundary for very high heterogeneity.

 
Impact of deviations from HWE
Adjustment for deviations from HWE changed the summary odds ratios by less than 10 percent in 33 of 42 meta-analyses. In the remaining nine meta-analyses, the change ranged between 10 percent and 31 percent, and the largest change was observed in a meta-analysis with only three studies. Overall, the summary adjusted odds ratio increased in 18 meta-analyses and decreased in 24 (figure 3). The pattern of changes was similar for allele-based meta-analyses and genotype-based meta-analyses (figure 3). In three meta-analyses of genotype-based contrasts, formal statistical significance of the postulated association was lost after adjustment for HWE deviations. Coronary artery disease was no longer associated with the L33P polymorphism of ITGB3 (ID 6 in table 1), schizophrenia was no longer associated with the Ser9Gly polymorphism of DRD3 (ID 29), and the same was true for myocardial infarction and the Q192R polymorphism of PON1 (ID 30). In all three instances, the p values in the original meta-analysis (without adjustment for HWE deviations) were relatively borderline (range, 0.02–0.04). There was no postulated gene-disease association that was nonsignificant in the unadjusted analysis but became significant after HWE adjustment.


Figure 3
View larger version (39K):
[in this window]
[in a new window]
  		FIGURE 3. Meta-analyses before and after adjustments for departures from the Hardy-Weinberg law. Each meta-analysis is preceded by the (operational) identification number (ID) given in table 1, the first author of the study and the year of publication (e.g., '98 = 1998), and the reference number. The lower set of 11 meta-analyses used allele-based contrasts. For each meta-analysis, summary odds ratios and their 95% confidence intervals (CIs) are depicted. *, meta-analyses in which formal statistical significance was lost after adjustment for deviations from Hardy-Weinberg equilibrium (HWE).

 
After adjustments for deviations from HWE, between-study heterogeneity, as expressed by I2, increased in 28 meta-analyses and decreased in eight (p < 0.001, adjusted for six ties) (figure 4). In 11 instances, the increase in I2 was more than 10 percent, whereas, in two meta-analyses, I2 decreased by more than 10 percent, after the adjustments (p = 0.013). Heterogeneity became formally statistically significant in seven instances (the MTHFR C677T polymorphism and neural tube defects (ID 3 in table 1); the FCGR2A R131H polymorphism and systemic lupus erythematosus (ID 10); the UCHL1 S18Y polymorphism and Parkinson's disease (ID 17); the PON1 Q192R polymorphism and myocardial infarction (ID 30); the MTHFR C677T polymorphism and venous thromboembolism (ID 37); the TGFBR1 (del(GCC)3) polymorphism and cancer (ID 38); and the TP53 P72R polymorphism and lung cancer (ID 39)) where it was previously not formally significant; the opposite change was never seen. All seven instances in which heterogeneity became formally statistically significant after the correction used genotype comparisons.


Figure 4
View larger version (19K):
[in this window]
[in a new window]
  		FIGURE 4. Extent of heterogeneity before and after adjustments for departures from the Hardy-Weinberg law for the controls. The layout is similar to that described for figure 2.

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
APPENDIX
 References
 
In this empirical investigation, we found that exclusion of HWE-violating studies may result in loss of the statistical significance of some postulated gene-disease associations and that adjustment for the magnitude of deviation from HWE may also have the same consequence for some other gene-disease associations. The two corrective measures seemed to affect different associations. Although the majority of the postulated gene-disease associations were not affected by these corrective measures, six of 23 gene-disease associations for which there was formally significant evidence lost their significance after exclusion of HWE-violating studies or adjustment for HWE deviations. In all of these instances, the results of the original meta-analysis were not highly statistically significant, and the original p values ranged between 0.007 and 0.04. Therefore, statistical significance was lost even though the change in the magnitude of the odds ratio was typically small with these corrective measures. None of the 19 nonsignificant associations became significant after we applied such HWE-related corrective measures. Exclusion of HWE-violating studies seemed to diminish modestly the between-study heterogeneity in some instances, whereas adjustment for HWE deviations tended to have the opposite effect.

Prior investigations seem to agree that significant deviations from HWE regarding genotype frequencies among healthy controls occur in approximately 10 percent of gene-disease association studies (14Go, 53Go–56Go). Only 5 percent would be expected by chance alone at the {alpha} = 0.05 significance level with adequately powered studies, and the actual expected percentage is probably even lower because several gene-disease association studies use very small, underpowered sample sizes (14Go). HWE may be violated because of genotyping error, chance, inbreeding, nonrandom mating, differential survival of marker carriers, genetic drifting, population stratification, or combinations of these reasons. Some of these effects may also occasionally act in different directions, canceling each other. Unfortunately, most gene-disease association studies report very limited information to provide any detailed insight into these potential problems. Thus, also at a meta-analysis level, typically there is no way to decipher for which of the above reasons the violation has occurred, except perhaps for population stratification under special conditions (57Go). For this reason, sensitivity analyses including and excluding the HWE-deviating studies have been recommended (58Go). However, most published meta-analyses systematically neglect such analyses (59Go). Adjustments for the magnitude of deviation from HWE are rarely performed in primary studies, and, to date, these corrections have not been introduced at all into meta-analyses, to our knowledge. Such adjustments are as useful to perform as exclusion of HWE-violating studies. The two corrective measures may offer complementary information.

Whenever HWE-related corrective measures resulted in loss of the significance of an association, the change in the estimates of the odds ratio was relatively small with these corrections. However, most effect sizes in genetic epidemiology are very modest anyway (3Go, 7Go–9Go). Formal statistical significance is routinely considered important for deciding whether or not the probed association is present. Our finding suggests that gene-disease associations should be pronounced with extra caution when p values are not much smaller than 0.05. Relatively minor deviations in the genotype frequencies could change whether or not formal significance is present. Thus, we recommend that meta-analyses and individual studies routinely examine whether their inferences are robust to accounting and correcting for HWE violations and deviations. When results are not robust, more data should be considered from well-designed studies. Moreover, the threshold for claiming statistical significance in gene-disease associations may need to shift toward smaller p values (60Go–63Go); however, no p-value threshold is likely to be perfect, and many methodologists are skeptical of p values. Alternatively, inferences should take into consideration not only the p value but also the prestudy probability that a postulated association is true and the potential for bias (64Go, 65Go).

The observed between-study heterogeneity for postulated gene-disease associations cannot be explained solely by the presence of HWE-violating studies. In fact, these studies seem to account for a small proportion of the observed heterogeneity. This finding may suggest that there are too few HWE-violating studies to make a difference in this regard, that formal HWE violation is an insensitive and/or unspecific marker of bias, that HWE tests are underpowered for most of the currently published case-control studies, or that there may be many other reasons that genetic association studies have heterogeneous results. Indeed, between-study heterogeneity in meta-analyses of gene-disease associations is typically substantial (66Go), and other tests, rather than simple HWE testing, may be more useful to dissect this heterogeneity (3Go, 7Go). It is prudent to exercise even greater caution when only a few studies are available, since early published data may be biased for other reasons (3Go, 66Go, 67Go).

After we adjusted for deviations from the Hardy-Weinberg law, studies became more heterogeneous among themselves, and, in seven meta-analyses, heterogeneity became formally significant. This finding is probably attributed mainly to the fact that, for genotype-based contrasts, the variance of each study's adjusted odds ratio becomes smaller after correcting for departures from HWE (22Go). Thus, correction for HWE deviations may make even more prominent the diversity in the results of studies on the same postulated association. The between-study heterogeneity observed in genetic epidemiology is usually larger compared with other research fields (7Go, 66Go). This heterogeneity is important to describe, quantify, and try to explain. Meta-analysis should aim at more than simply arriving to a grand mean (68Go), and dissection of between-study heterogeneity can provide some valuable information.

Some limitations of the present empirical evaluation should be discussed. First, the type of genetic contrast chosen may have affected some of the analyses we performed. For this reason, we used an a priori algorithm that had been already agreed on in previous research (3Go) to avoid subjectivity in choosing genetic contrasts. Second, only those meta-analyses for which genotype-level information was available were selected. This limitation was unavoidable, however, since one cannot rely on the assertions of the primary authors about HWE testing, because these assertions often do not represent the data accurately (14Go). Third, even though almost 600 studies were targeted, this number still represents only a small proportion of the gene-disease association studies conducted to date. However, there is no strong reason to believe that these studies and their meta-analyses would suffer from selection biases that would invalidate the aforementioned findings. Finally, the extent to which the HWE-related problems were due to genotyping error could not be probed. Doing so would require retesting samples from the analyzed studies and would be infeasible given the wide diversity of teams involved in these studies. The suggestion of correction for HWE violations might not seem very practical in genotype-based studies if individual studies have verified that the HWE violations were not due to genotyping error. However, apart from the fact that few—if any—studies can claim immunity to genotypic errors, departure from HWE even due to chance may introduce a statistical bias that would especially affect allele-based contrasts (69Go). Thus, correction for departures from HWE should be applicable even when the probability for substantial genotyping error is low. We should also acknowledge that the proposed corrections may be more complex if other covariates, and/or matching of cases and controls, have to be taken into account in a meta-analysis. Such issues might be addressed by using hierarchic meta-regression analyses in a Bayesian framework that would also take into account departures from HWE. However, the vast majority of case-control genetic association studies still present primarily and/or exclusively unmatched data without covariate adjustment.

Allowing for these caveats, we conclude that HWE should be routinely and transparently assessed in gene-disease association studies and their meta-analyses. Simple statistical testing for significant violations of the Hardy-Weinberg law may not suffice, and the data should be scrutinized by also taking into account nonsignificant deviations from HWE. Instructions and guidance to authors of gene-disease association studies and meta-analyses thereof may need to reinforce attention to HWE issues. Discrepant results in these analyses do not necessarily mean that postulated associations should be dismissed, but they should hint at the need for more evidence and validation.


   APPENDIX
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
References
 
To correct for departures from HWE for the controls for genotype-based contrasts (22Go), let G1 and G2 be the genotype groups that are contrasted in an association study of a biallelic locus (alleles A and {alpha}) with a disease (by genotype groups, we refer to genotypes: AA, Aa, or aa; or combinations of genotypes: AA + Aa, Aa, + aa, etc.). The Lathrop correction for deviations from HWE in the controls calculates the adjusted odds ratio (ORadj) by using the HWE-predicted genotype counts in the control group (Formula i being the genotype group index) instead of the observed ones: Formula

The variance of the natural logarithm of the adjusted odds ratio would then become Formula where Formula is an estimate of the variance of the HWE-predicted control counts depending on the specific contrast G1 vs G2. More specifically, if pA, pa are the corresponding allele frequencies in the controls, and NA and Na is the total number of alleles in the controls, then Formula is given by the following formulae for the different types of contrasts:

Formula


   ACKNOWLEDGMENTS
 
Conflict of interest: none declared.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 

  1. Lander ES, Schork NJ. Genetic dissection of complex traits. Science 1994;265:2037–48.[Abstract/Free Full Text]
  2. Khoury MJ, Little J. Human genome epidemiologic reviews: the beginning of something HuGE. Am J Epidemiol 2000;151:2–3.[Free Full Text]
  3. Ioannidis JP, Ntzani EE, Trikalinos TA, et al. Replication validity of genetic association studies. Nat Genet 2001;29:306–9.[CrossRef][Web of Science][Medline]
  4. Hirschhorn JN, Lohmueller K, Byrne E, et al. A comprehensive review of genetic association studies. Genet Med 2002;4:45–61.[Web of Science][Medline]
  5. Little J, Khoury MJ, Bradley L, et al. The human genome project is complete. How do we develop a handle for the pump? Am J Epidemiol 2003;157:667–73.[Free Full Text]
  6. Easterbrook PJ, Berlin JA, Gopalan R, et al. Publication bias in clinical research. Lancet 1991;337:867–72.[CrossRef][Web of Science][Medline]
  7. Ioannidis JP, Trikalinos TA, Ntzani EE, et al. Genetic associations in large versus small studies: an empirical assessment. Lancet 2003;361:567–71.[CrossRef][Web of Science][Medline]
  8. Ioannidis JP, Ntzani EE, Trikalinos TA. ‘Racial’ differences in genetic effects for complex diseases. Nat Genet 2004;36:1312–18.[CrossRef][Web of Science][Medline]
  9. Lohmueller KE, Pearce CL, Pike M, et al. Meta-analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nat Genet 2003;33:177–82.[CrossRef][Web of Science][Medline]
  10. Khoury MJ, Beaty TH, Cohen BH, eds. Fundamentals of genetic epidemiology. New York, NY: Oxford University Press, 1993.
  11. Mitchell AA, Cutler DJ, Chakravarti A. Undetected genotyping errors cause apparent overtransmission of common alleles in the transmission/disequilibrium test. Am J Hum Genet 2003;72:598–610.[CrossRef][Web of Science][Medline]
  12. Hosking L, Lumsden S, Lewis K, et al. Detection of genotyping errors by Hardy-Weinberg equilibrium testing. Eur J Hum Genet 2004;12:395–9.[CrossRef][Web of Science][Medline]
  13. Bardoczy Z, Gyorffy B, Kocsis I, et al. Re-calculated Hardy-Weinberg values in papers published in Atherosclerosis between 1995 and 2003. Atherosclerosis 2004;173:141–3.[CrossRef][Web of Science][Medline]
  14. Salanti G, Amountza G, Ntzani EE, et al. Hardy-Weinberg equilibrium in genetic association studies: an empirical evaluation of reporting, deviations, and power. Eur J Hum Genet 2005;13:840–8.[CrossRef][Web of Science][Medline]
  15. Shoemaker J, Painter I, Weir BS. A Bayesian characterization of Hardy-Weinberg disequilibrium. Genetics 1998;149:2079–88.[Abstract/Free Full Text]
  16. Emigh T. A comparison of tests for Hardy-Weinberg equilibrium. Biometrics 1980;36:627–42.[CrossRef][Web of Science]
  17. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177–88.[CrossRef][Web of Science][Medline]
  18. Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997;127:820–6.[Abstract/Free Full Text]
  19. Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327:557–60.[Free Full Text]
  20. Cappelleri JC, Ioannidis JP, Schmid CH, et al. Large trials vs meta-analysis of smaller trials: how do their results compare? JAMA 1996;276:1332–8.[Abstract/Free Full Text]
  21. Schaid DJ, Jacobsen SJ. Biased tests of association: comparisons of allele frequencies when departing from Hardy-Weinberg proportions. Am J Epidemiol 1999;149:706–11.[Abstract/Free Full Text]
  22. Lathrop MG. Estimating genotype relative risks. Tissue Antigens 1983;22:160–6.[Web of Science][Medline]
  23. Noble EP. The D2 dopamine receptor gene: a review of association studies in alcoholism and phenotypes. Alcohol 1998;16:33–45.[CrossRef][Web of Science][Medline]
  24. Sharma P. Meta-analysis of the ACE gene in ischaemic stroke. J Neurol Neurosurg Psychiatry 1998;64:227–30.[Abstract/Free Full Text]
  25. Botto LD, Yang Q. 5,10-Methylenetetrahydrofolate reductase gene variants and congenital anomalies: a HuGE review. Am J Epidemiol 2000;151:862–77.[Abstract/Free Full Text]
  26. Hinney A, Remschmidt H, Hebebrand J. Candidate gene polymorphisms in eating disorders. Eur J Pharmacol 2000;410:147–59.[CrossRef][Web of Science][Medline]
  27. Mizuta I, Mizuta E, Yamasaki S, et al. Meta-analysis of polymorphism of the catechol-O-methyltransferase gene in relation to the etiology of Parkinson's disease in Japan. Mov Disord 2000;15:1013–14.[CrossRef][Web of Science][Medline]
  28. Di Castelnuovo A, de Gaetano G, Donati MB, et al. Platelet glycoprotein receptor IIIa polymorphism PLA1/PLA2 and coronary risk: a meta-analysis. Thromb Haemost 2001;85:626–33.[Web of Science][Medline]
  29. Efstathiadou Z, Tsatsoulis A, Ioannidis JP. Association of collagen Ialpha 1 Sp1 polymorphism with the risk of prevalent fractures: a meta-analysis. J Bone Miner Res 2001;16:1586–92.[CrossRef][Web of Science][Medline]
  30. Schena FP, D'Altri C, Cerullo G, et al. ACE gene polymorphism and IgA nephropathy: an ethnically homogeneous study and a meta-analysis. Kidney Int 2001;60:732–40.[CrossRef][Web of Science][Medline]
  31. Trikalinos TA, Karassa FB, Ioannidis JP. Meta-analysis of the association between low-affinity Fcgamma receptor gene polymorphisms and hematologic and autoimmune disease. (Letter). Blood 2001;98:1634–5.[Free Full Text]
  32. Bonnici F, Keavney B, Collins R, et al. Angiotensin converting enzyme insertion or deletion polymorphism and coronary restenosis: meta-analysis of 16 studies. BMJ 2002;325:517–20.[Abstract/Free Full Text]
  33. Feyler A, Voho A, Bouchardy C, et al. Point: myeloperoxidase-463G -> a polymorphism and lung cancer risk. Cancer Epidemiol Biomarkers Prev 2002;11:1550–4.[Abstract/Free Full Text]
  34. Karassa FB, Trikalinos TA, Ioannidis JP, et al. Role of the Fcgamma receptor IIa polymorphism in susceptibility to systemic lupus erythematosus and lupus nephritis: a meta-analysis. Arthritis Rheum 2002;46:1563–71.[CrossRef][Web of Science][Medline]
  35. Karassa FB, Bijl M, Davies KA, et al. Role of the Fcgamma receptor IIA polymorphism in the antiphospholipid syndrome: an international meta-analysis. Arthritis Rheum 2003;48:1930–8.[CrossRef][Web of Science][Medline]
  36. Karassa FB, Trikalinos TA, Ioannidis JP, et al. The Fc gamma RIIIA-F158 allele is a risk factor for the development of lupus nephritis: a meta-analysis. Kidney Int 2003;63:1475–82.[CrossRef][Web of Science][Medline]
  37. Ray JG, Shmorgun D, Chan WS. Common C677T polymorphism of the methylenetetrahydrofolate reductase gene and the risk of venous thromboembolism: meta-analysis of 31 studies. Pathophysiol Haemost Thromb 2002;32:51–8.[CrossRef][Web of Science][Medline]
  38. Anguelova M, Benkelfat C, Turecki G. A systematic review of association studies investigating genes coding for serotonin receptors and the serotonin transporter: II. Suicidal behavior. Mol Psychiatry 2003;8:646–53.[CrossRef][Web of Science][Medline]
  39. Jonsson EG, Flyckt L, Burgert E, et al. Dopamine D3 receptor gene Ser9Gly variant and schizophrenia: association study and meta-analysis. Psychiatr Genet 2003;13:1–12.[CrossRef][Web of Science][Medline]
  40. Kehoe PG, Katzov H, Feuk L, et al. Haplotypes extending across ACE are associated with Alzheimer's disease. Hum Mol Genet 2003;12:859–67.[Abstract/Free Full Text]
  41. Kosmas IP, Tatsioni A, Ioannidis JP. Association of Leiden mutation in factor V gene with hypertension in pregnancy and pre-eclampsia: a meta-analysis. J Hypertens 2003;21:1221–8.[CrossRef][Web of Science][Medline]
  42. Kosmas IP, Tatsioni A, Ioannidis JP. Association of C677T polymorphism in the methylenetetrahydrofolate reductase gene with hypertension in pregnancy and pre-eclampsia: a meta-analysis. J Hypertens 2004;22:1655–62.[CrossRef][Web of Science][Medline]
  43. Matakidou A, Eisen T, Houlston RS. TP53 polymorphisms and lung cancer risk: a systematic review and meta-analysis. Mutagenesis 2003;18:377–85.[Abstract/Free Full Text]
  44. Nielsen EM, Hansen L, Carstensen B, et al. The E23K variant of Kir6.2 associates with impaired post-OGTT serum insulin response and increased risk of type 2 diabetes. Diabetes 2003;52:573–7.[Abstract/Free Full Text]
  45. Ntais C, Polycarpou A, Ioannidis JP. Association of the CYP17 gene polymorphism with the risk of prostate cancer: a meta-analysis. Cancer Epidemiol Biomarkers Prev 2003;12:120–6.[Abstract/Free Full Text]
  46. Ntais C, Polycarpou A, Ioannidis JP. SRD5A2 gene polymorphisms and the risk of prostate cancer: a meta-analysis. Cancer Epidemiol Biomarkers Prev 2003;12:618–24.[Abstract/Free Full Text]
  47. Ntais C, Polycarpou A, Ioannidis JP. Vitamin D receptor gene polymorphisms and risk of prostate cancer: a meta-analysis. Cancer Epidemiol Biomarkers Prev 2003;12:1395–402.[Abstract/Free Full Text]
  48. Ntais C, Polycarpou A, Ioannidis JP. Meta-analysis of the association of the Cathepsin D Ala224Val gene polymorphism with the risk of Alzheimer's disease: a HuGE gene-disease association review. Am J Epidemiol 2004;159:527–36.[Abstract/Free Full Text]
  49. Sethi AA, Nordestgaard BG, Tybjaerg-Hansen A. Angiotensinogen gene polymorphism, plasma angiotensinogen, and risk of hypertension and ischemic heart disease: a meta-analysis. Arterioscler Thromb Vasc Biol 2003;23:1269–75.[Abstract/Free Full Text]
  50. Maraganore DM, Lesnick TG, Elbaz A, et al. UCHL1 is a Parkinson's disease susceptibility gene. Ann Neurol 2004;55:512–21.[CrossRef][Web of Science][Medline]
  51. Pasche B, Kaklamani V, Hou N, et al. TGFBR1*6A and cancer: a meta-analysis of 12 case-control studies. (Letter). J Clin Oncol 2004;22:756–8.[Free Full Text]
  52. Wheeler JG, Keavney BD, Watkins H, et al. Four paraoxonase gene polymorphisms in 11212 cases of coronary heart disease and 12786 controls: meta-analysis of 43 studies. Lancet 2004;363:689–95.[CrossRef][Web of Science][Medline]
  53. Gyorffy B, Kocsis I, Vasarhelyi B. Biallelic genotype distributions in papers published in Gut between 1998 and 2003: altered conclusions after recalculating the Hardy-Weinberg equilibrium. (Letter). Gut 2004;53:614–15.[Free Full Text]
  54. Gyorffy B, Kocsis I, Vasarhelyi B. Missed calculations and new conclusions: re-calculation of genotype distribution data published in Journal of Investigative Dermatology, 1998–2003. (Letter). J Invest Dermatol 2002;122:644–6.[CrossRef]
  55. Kocsis I, Gyorffy B, Nemeth E, et al. Examination of Hardy-Weinberg equilibrium in papers of Kidney International: an underused tool. Kidney Int 2004;65:1956–8.[CrossRef][Web of Science][Medline]
  56. Kocsis I, Vasarhelyi B, Gyorffy A, et al. Reanalysis of genotype distributions published in Neurology between 1999 and 2002. Neurology 2004;63:357–8.[Abstract/Free Full Text]
  57. Pritchard JK, Stephens M, Rosenberg NA, et al. Association mapping in structured populations. Am J Hum Genet 2000;67:170–81.[CrossRef][Web of Science][Medline]
  58. Thakkinstian A, McElduff P, D'Este C, et al. A method for meta-analysis of molecular association studies. Stat Med 2004;24:1291–306.
  59. Attia J, Thakkinstian A, D'Este C. Meta-analyses of molecular association studies: methodologic lessons for genetic epidemiology. J Clin Epidemiol 2003;56:297–303.[CrossRef][Web of Science][Medline]
  60. Risch N, Merikangas K. The future of genetic studies of complex human diseases. Science 1996;273:1516–17.[Abstract/Free Full Text]
  61. Altshuler D, Kruglyak L, Lander E. Genetic polymorphisms and disease. (Letter). N Engl J Med 1998;338:1626.[Free Full Text]
  62. Dahlman I, Eaves IA, Kosoy R, et al. Parameters for reliable results in genetic association studies in common disease. Nat Genet 2002;30:149–50.[CrossRef][Web of Science][Medline]
  63. Lander E, Kruglyak L. Genetic dissection of complex traits: guidelines for interpreting and reporting linkage results. Nat Genet 1995;11:241–7.[CrossRef][Web of Science][Medline]
  64. Wacholder S, Chanock S, Garcia-Closas M, et al. Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J Natl Cancer Inst 2004;96:434–42.[Abstract/Free Full Text]
  65. Ioannidis JP. Why most published research findings are false. PLoS Med 2005;2(8):e124. (DOI: 10.1371/journal.pmed.0020124).[CrossRef][Medline]
  66. Ioannidis JP, Trikalinos TA. Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol 2005;58:543–9.[CrossRef][Web of Science][Medline]
  67. Trikalinos TA, Ntzani EE, Contopoulos-Ioannidis DG, et al. Establishment of genetic associations for complex diseases is independent of early study findings. Eur J Hum Genet 2004;12:762–9.[CrossRef][Web of Science][Medline]
  68. Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 1998;351:123–7.[CrossRef][Web of Science][Medline]
  69. Sasieni PD. From genotypes to genes: doubling the sample size. Biometrics 1997;53:1253–61.[CrossRef][Web of Science][Medline]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 J Geriatr Psychiatry NeurolHome page
Hua Liu, Ming Liu, Wei Li, Bo Wu, S.-h. Zhang, Yuan Fang, and Yi Wang
Association of ACE I/D Gene Polymorphism With Vascular Dementia: A Meta-Analysis
J Geriatr Psychiatry Neurol, March 1, 2009; 22(1): 10 - 22.
[Abstract] [PDF]

Home page
 ANN INTERN MEDHome page
J. Little, J. P.T. Higgins, J. P.A. Ioannidis, D. Moher, F. Gagnon, E. von Elm, M. J. Khoury, B. Cohen, G. Davey-Smith, J. Grimshaw, et al.
STrengthening the REporting of Genetic Association Studies (STREGA): An Extension of the STROBE Statement
Ann Intern Med, February 3, 2009; 150(3): 206 - 215.
[Abstract] [Full Text] [PDF]

Home page
 JNCI J Natl Cancer InstHome page
P. Vineis, M. Manuguerra, F. K. Kavvoura, S. Guarrera, A. Allione, F. Rosa, A. Di Gregorio, S. Polidoro, F. Saletta, J. P. A. Ioannidis, et al.
A Field Synopsis on Low-Penetrance Variants in DNA Repair Genes and Cancer Susceptibility
J Natl Cancer Inst, January 7, 2009; 101(1): 24 - 36.
[Abstract] [Full Text] [PDF]

Home page
 Mol Hum ReprodHome page
P. G. Bagos
Plasminogen activator inhibitor-1 4G/5G and 5,10-methylene-tetrahydrofolate reductase C677T polymorphisms in polycystic ovary syndrome
Mol. Hum. Reprod., January 1, 2009; 15(1): 19 - 26.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
M.-h. Chang, M. L. Lindegren, M. A. Butler, S. J. Chanock, N. F. Dowling, M. Gallagher, R. Moonesinghe, C. A. Moore, R. M. Ned, M. R. Reichler, et al.
Prevalence in the United States of Selected Candidate Gene Variants: Third National Health and Nutrition Examination Survey, 1991-1994
Am. J. Epidemiol., January 1, 2009; 169(1): 54 - 66.
[Abstract] [Full Text] [PDF]

Home page
 Neuro OncolHome page
S. Lonn, N. Rothman, W. R. Shapiro, H. A. Fine, R. G. Selker, P. M. Black, J. S. Loeffler, A. A. Hutchinson, and P. D. Inskip
Genetic variation in insulin-like growth factors and brain tumor risk
Neuro-oncol, August 1, 2008; 10(4): 553 - 559.
[Abstract] [Full Text] [PDF]

Home page
 J. Med. Genet.Home page
J J. Galan, B Buch, S Pedrinaci, P Jimenez-Gamiz, A Gonzalez, M Serrano-Rios, A Salinas, M d. C. Rivero, L M Real, J L Royo, et al.
Identification of a 2244 base pair interstitial deletion within the human ESR1 gene in the Spanish population
J. Med. Genet., July 1, 2008; 45(7): 420 - 424.
[Abstract] [Full Text] [PDF]

Home page
 Int J EpidemiolHome page
C. Minelli, J. R Thompson, K. R Abrams, A. Thakkinstian, and J. Attia
How should we use information about HWE in the meta-analyses of genetic association studies?
Int. J. Epidemiol., February 1, 2008; 37(1): 136 - 146.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
L. Smith, M. Watson, S. Gates, D. Ball, and D. Foxcroft
Meta-Analysis of the Association of the Taq1A Polymorphism with the Risk of Alcohol Dependency: A HuGE Gene-Disease Association Review
Am. J. Epidemiol., January 15, 2008; 167(2): 125 - 138.
[Abstract] [Full Text] [PDF]

Home page
 Reproductive SciencesHome page
K. J. A. F. van Kaam, A. Romano, G. A. J. Dunselman, and P. G. Groothuis
Transforming Growth Factor {beta}1 Gene Polymorphism 509C/T in Deep Infiltrating Endometriosis
Reproductive Sciences, May 1, 2007; 14(4): 367 - 373.
[Abstract] [PDF]

Home page
 Int J EpidemiolHome page
M. J Khoury, J. Little, M. Gwinn, and J. P. Ioannidis
On the synthesis and interpretation of consistent but weak gene-disease associations in the era of genome-wide association studies
Int. J. Epidemiol., April 1, 2007; 36(2): 439 - 445.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
S. Gilbody, S. Lewis, and T. Lightfoot
Methylenetetrahydrofolate Reductase (MTHFR) Genetic Polymorphisms and Psychiatric Disorders: A HuGE Review
Am. J. Epidemiol., January 1, 2007; 165(1): 1 - 13.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
A. Thakkinstian, S. Bowe, M. McEvoy, W. Smith, and J. Attia
Association between Apolipoprotein E Polymorphisms and Age-related Macular Degeneration: A HuGE Review and Meta-Analysis
Am. J. Epidemiol., November 1, 2006; 164(9): 813 - 822.
[Abstract] [Full Text] [PDF]

Home page
 Cancer Epidemiol. Biomarkers Prev.Home page
F. Kamangar, C. Cheng, C. C. Abnet, and C. S. Rabkin
Interleukin-1B Polymorphisms and Gastric Cancer Risk--A Meta-analysis.
Cancer Epidemiol. Biomarkers Prev., October 1, 2006; 15(10): 1920 - 1928.
[Abstract] [Full Text] [PDF]

Home page
 Cancer Epidemiol. Biomarkers Prev.Home page
T. V. Pereira, M. Rudnicki, A. C. Pereira, M. S. Pombo-de-Oliveira, and R. F. Franco
5,10-Methylenetetrahydrofolate Reductase Polymorphisms and Acute Lymphoblastic Leukemia Risk: A Meta-analysis.
Cancer Epidemiol. Biomarkers Prev., October 1, 2006; 15(10): 1956 - 1963.
[Abstract] [Full Text] [PDF]

Home page
 Hum Mol GenetHome page
A. Thakkinstian, P. Han, M. McEvoy, W. Smith, J. Hoh, K. Magnusson, K. Zhang, and J. Attia
Systematic review and meta-analysis of the association between complementary factor H Y402H polymorphisms and age-related macular degeneration
Hum. Mol. Genet., September 15, 2006; 15(18): 2784 - 2790.
[Abstract] [Full Text] [PDF]

Home page
 Cancer Epidemiol. Biomarkers Prev.Home page
M. C. Camargo, R. Mera, P. Correa, R. M. Peek Jr., E. T.H. Fontham, K. J. Goodman, M. B. Piazuelo, L. Sicinschi, J. Zabaleta, and B. G. Schneider
Interleukin-1{beta} and Interleukin-1 Receptor Antagonist Gene Polymorphisms and Gastric Cancer: A Meta-analysis.
Cancer Epidemiol. Biomarkers Prev., September 1, 2006; 15(9): 1674 - 1687.
[Abstract] [Full Text] [PDF]

Home page
 JAMAHome page
D. M. Maraganore, M. de Andrade, A. Elbaz, M. J. Farrer, J. P. Ioannidis, R. Kruger, W. A. Rocca, N. K. Schneider, T. G. Lesnick, S. J. Lincoln, et al.
Collaborative analysis of alpha-synuclein gene promoter variability and Parkinson disease.
JAMA, August 9, 2006; 296(6): 661 - 670.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
163/4/300    most recent
kwj046v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (31)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Trikalinos, T. A.
Right arrow Articles by Ioannidis, J. P. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Trikalinos, T. A.
Right arrow Articles by Ioannidis, J. P. A.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?