Am J Epidemiol 2003; 157:364-375.
Copyright © 2003 by Johns
Hopkins Bloomberg School of Public Health
PRACTICE OF EPIDEMIOLOGY |
Statistical Analysis of Correlated Data Using Generalized Estimating Equations: An Orientation
1 Department of Epidemiology and Biostatistics, Faculty of Medicine, McGill University, Montreal, Quebec, Canada.
2 Division of Clinical Epidemiology, Royal Victoria Hospital, Montreal, Quebec, Canada.
3 Division of Epidemiology and Biostatistics, Department of Epidemiology and Social Medicine, Albert Einstein College of Medicine of Yeshiva University, Bronx, NY.
4 Department of Family Medicine and Community Health, School of Medicine, Tufts University, Boston, MA.
Received for publication January 7, 2000; accepted for publication August 7, 2002.
| ABSTRACT |
|---|
|
|
|---|
The method of generalized estimating equations (GEE) is often used to analyze longitudinal and other correlated response data, particularly if responses are binary. However, few descriptions of the method are accessible to epidemiologists. In this paper, the authors use small worked examples and one real data set, involving both binary and quantitative response data, to help end-users appreciate the essence of the method. The examples are simple enough to see the behind-the-scenes calculations and the essential role of weighted observations, and they allow nonstatisticians to imagine the calculations involved when the GEE method is applied to more complex multivariate data.
correlation; epidemiologic methods; generalized estimating equation; longitudinal studies; odds ratio; statistics
Abbreviations: Abbreviation: GEE, generalized estimating equations.
| INTRODUCTION |
|---|
|
|
|---|
The generalized estimating equations (GEE) (1, 2) method, an extension of the quasi-likelihood approach (3), is being increasingly used to analyze longitudinal (4) and other (5) correlated data, especially when they are binary or in the form of counts. We are aware of only two articles which try to make the GEE approach more accessible to nonstatisticians. One focuses on software (6). The other, an excellent expository article (5) covering several approaches to correlated data, has limited coverage of GEE. Examples in most texts and manuals are too extensive, and the treatment too theoretical, to allow end-users to follow the calculations or fully appreciate the principles behind them. In this paper, we attempt to redress this. To illustrate the ideas, we use the data shown in table 1. They consist of the age- and sex-standardized heights (and data on the covariates gender and socioeconomic status) of 144 children in a sample of 54 randomly selected households in Mexico (7).
|
Textbooks all advise researchers not to treat observations from the same household (or "cluster") as if they were independent and thus not to calculate standard errors using n = 144 as the sample size. For example, Colton (8, pp. 4143) warns against being misled by "great masses of observations, which upon closer scrutiny, may often vanish," and he uses as an example an n of 800 blood pressure measurements10 taken each week over an 8-week treatment course, in 10 patients! He stresses that "appropriate conclusions regarding the drugs effect rely on subject-to-subject variation, so that the sample size of 10 subjects is crucial to such analysis." However, few texts explain how one is to properly use all 800 (or, in our example, 144) data points, or how much each observation contributes statistically.
Although some articles do discuss how much statistical information is obtainable from observations on paired organs (9) or individuals in clusters such as classrooms or physicians practices (10), investigators often take a conservative approach. In one example, where all eligible children in a household were randomized to the same treatment (11), statistics were computed as if the observations were independent but standard errors were based on the numbers of households. In another (12), where one fourth of the subjects had a sibling in the study, the authors excluded the data obtained from one of the two siblings.
In this expository article, we show how the GEE approach uses weighted combinations of observations to extract the appropriate amount of information from correlated data. We first motivate and introduce the approach using hand calculations on small hypothetical data sets. We use households as clusters, with the letter "h" (household) as a subscript. We use the Greek letters µ and
and the uppercase letters P, B, and R when referring to a parameter (a mean, standard deviation, proportion, or regression or correlation coefficient); and we use the symbol
and the lowercase letters p, b, and r for the corresponding statistic (empirical value, calculated from a sample) which serves as an estimate of the parameter.
| ELEMENTS |
|---|
|
|
|---|
Variability of statistics formed from weighted sums or weighted averages of observationsthe general case
The instability of a statistic is measured by its variance. Many statistics involve weighted sums of observations or random variables; weights that add to 1 produce weighted averages. In the general case, the variance of a weighted sum of n random variables y1 to yn is a sum of n2 products. These involve 1) the n weights, w1 to wn; 2) the n standard deviations,
1 to
n, of the random variables; and 3) the n x n matrix of pairwise correlations, R1,1 to Rn,n, of the random variables. As figure 1 illustrates, the variance of a weighted sum or average can be conveniently computed by placing w1 to wn and
1 to
n along both the row and column margins of the n x n correlation matrix, forming the product
|
wrow x wcolumn x
row x
column x Rrow,column
for each {row, column} combination, and then summing these products over the n2 row-column combinations.
For the remainder of this section, we will assume that the
s are all equal.
Variability of statistics derived from uncorrelated observations
When the observations are uncorrelated, the off-diagonal elements in the correlation matrix are zero. If each of the n weights equals 1/n, then the weighted sum is the mean,
. Its variance (the sum of the diagonal elements in part b of figure 1) is thus
,
yielding the familiar formula SD[
] =
/
.
With equal statistical weights of 1 each, the variance of the simple sum is Var[
y] = n
2, so that SD[
y] =
. Although our main example involves "physical" heights and "statistical" weights, a side example is instructive. Assume that the "physical" weights of elevator-taking adults vary from person to person by, for example,
= 10 kg. Then elevators of 16 persons each (i.e., n = 16), randomly chosen from among these, will vary from load to load with a standard deviation of (only!)
(10) = 40 kg, while the average per person in each load of 16 will vary with a standard deviation of only 10/
= 2.5 kg.
Variability of statistics derived from correlated observations
In the elevator example, the "
/
" and "
" laws for the variability of the two statistics do not hold if the variable of interest on sampled individuals tends to be similar from one individual to the other ("co-related")for example, if elevators are sometimes used by professional football teams and sometimes by ballet dance classes. The variance of a weighted combination of such observations now involvesin addition to the 1s on the diagonalthe pairwise nonzero off-diagonal elements of the correlation matrix.
When the ys of individuals in a cluster are positively correlated, as is typical, the additional off-diagonal elements in part b of figure 1 make the standard deviation of the unweighted average
greater than
/
.
Preamble to GEE: optimal combination of correlated observations
Suppose, for simplicity, that households have either one or two children and that the mean (µ) and standard deviation (
) of the variable being measured are the same in both types of households (in some applications (see Hoffman et al. (13), p. 440), µ may vary systematically with cluster size, but that situation will not be considered here). Let the correlation of measurements within two-child households be R. Consider the estimation of µ using a measurement on each of three children (n = 3), one from a randomly chosen single-child household and two from a two-child household. The 3 x 3 correlation matrix (figure 2) for the three ys is made up of a 1 x 1 matrix for the response from the singleton, a 2 x 2 matrix for the two responses from the two siblings, and zeroes for pairs of responses from unrelated children. The ys for some actual pairs of unrelated children will both be above or below µ, but on average, across all possible such pairs, the expected product of deviations is zero.
|
The first three rows of figure 3 list different possible estimators of µeach one a different weighted average of the three random variables. The first is the "straight" average of the three observations, using weights of one third each. The second estimator discards one of the related observations. The third uses all three, first creating an average of the two related observations and then averaging it and the unrelated observation.
|
Since all three are "valid" (unbiased) estimates of µ, one can use their relative precision to choose among them. The variance of each (i.e., over all possible such samples of three) is given by
row
columnwrowwcolumnRrow,column
2, where summation is over all nine pairs. Since four of these nine pairwise correlations, and thus the products involving them, are zero, and two others are identical, the variance simplifies to that shown in the third footnote of figure 3. The different sets of weights lead to the different variances shown in the third column of figure 3. From these, a number of lessons emerge: The greater the correlation, the greater the variability of the estimator that gives a weight of one third to each observation (first row); unless there is perfect correlation, the estimator that discards one of the two correlated observations is more variable (i.e., less "efficient") than the others; and the estimator formed by averaging the two correlated observations and then averaging this with the other observation (third row) is less variable than the others in high-correlation situations but more variable than the others in low-correlation situations. For any given R, there is a less variable estimator than the three considered. Suppose that, relative to a weight of 1 for the observation on the singleton, the weight for the y for each sibling is w, yielding the weighted average
One can show that its variance,
2(1 + 2w2 + 2Rw2)/(1 + 2w)2, is lowest when w = 1/(1 + R). For example, if R = 0.5, the optimal (relative) weights are in the ratio 1:(2/3):(2/3), and the variance is (3/7)
2, smaller than that of the competitors.
More generally, suppose the sample of n consists of several sets of children from 1-, 2-, ..., k-child households, with the same µ and the same pairwise within-household correlation R in all households, regardless of size. If the responses are ordered by household, then the n x n correlation matrix consists of several repetitions of various "block-diagonal" patterns, as in figure 2. One can show by calculus that the optimal weights for combining the responses of individual children from households of sizes 1, 2, 3, ..., k are 1, 1/(1 + R), 1/(1 + 2R), ..., 1/(1 + {k 1}R). These values can be obtained by summing the entries in any row or column of the inverses of the 1 x 1, 2 x 2, ..., k x k submatrices in the overall n x n block-diagonal matrix used in the GEE equations (see next subsection).
With data from paired organs, all "clusters" are of size k = 2. Rosner and Milton (9) illustrate this idea of "effective sample size" using responses of a persons left and right eyes to the same treatment: If these have a correlation of 0.54, then 200 eyes, two from each of 100 persons, contribute the "statistical equivalent" of one-eye contributions from each of 130 persons (200 x 1/(1 + 0.54) = 130). The closer the correlation is to 1, the closer the effective sample size is to 100.
Estimation by GEE: the "EE" in GEE
In the n = 3 example in figures 2 and 3, consisting of just one household of size k = 1 and one of size k = 2, each y is a separate legitimate (unbiased) estimator,
, of µ (the circumflex or "hat" over µ denotes an estimate of it, calculated from data). As was the practice in the pre-least-squares era (14), one can combine the three separate estimating equations: ysingleton
= 0, ysib1
= 0, and ysib2
= 0, using the weights wsingleton, wsib1, and wsib2, to obtain a single estimating equation
In this simple case,
.
In this didactic example, the value of R used to construct the weights was considered "known"; in practice, it must be estimated, along with µ. The process is illustrated in figure 4, using a total of five observations (n = 5) from two clusters. Beginning with R = 0, one calculates five weights and, from them, an estimate of µ; from the degree of similarity of the within-cluster residuals, one obtains a new estimate, r, of R. The cycle is repeated until the estimates stabilizethat is, until "convergence" is achieved.
|
The estimating equation for the parameter µ has an obvious form. Equations for multiple regression parametersrepresenting absolute or relative differences in means, proportions, and ratesare formed by adapting the (iteratively re)weighted least squares equations used to obtain maximum likelihood parameter estimates from uncorrelated responses (3).
Estimation of a proportion (or odds) rather than a mean: the "G" in GEE
Figure 5 shows the GEE estimation of the expected proportion P from 0/3 and 4/5 positive responses in eight subjects in two households. The weights are 1/(1 + 2R) and 1/(1 + 4R) for the individuals in households of size three and five, respectively. The final estimate of P is p = 0.42, corresponding to r = 0.45. It is a weighed average of the eight 0s and 1s, with weights of 1/(1 + 2r) = 0.53 for each of the three responses in household 1 and 1/(1 + 4r) = 0.36 for each of the five responses in household 2; that is,
|
The sum of the eight weights, 0.53 each for the three persons in household 1 and 0.36 each for the five persons in household 2, can be viewed as the "effective" sample size of 3.39. Estimation of logit[P] = log[P/(1 P)] involves the same core calculations.
If P is different for different covariate patterns or strata, then the "unit" variance
2 = P(1 P) is no longer homogeneous. Nonconstant variances can be allowed for by incorporating a function of
2 into the weight for each observation (this is the basis of the iteratively reweighted least squares algorithm used with the usual logistic regression for uncorrelated responses).
Indeed, using different weights for each of n uncorrelated outcomes allows a unified approach to the maximum likelihood estimation of a family of "generalized linear models" (15, 16). Parameters are fitted by minimizing the weighted sum of squared residuals, using functions of the
2s as weights. For binomial and Poisson responses, where
2 is a function of the mean, weights are reestimated after each iteration. With GLIM software (17), Wacholder (18) illustrated how the risk difference, risk ratio, and odds ratio are estimated using the identity, log, and logit "links," respectively. This unified approach to uncorrelated responses has since become available in most other statistical packages. GEE implementations for correlated data use this same unified approach but use a quasi-likelihood rather than a full likelihood approach (3). Since correct specification of the mean and variance functions is sufficient for unbiased estimates, the model used does not fully specify the distribution of the responses in each cluster.
Standard errors: model-based or data-based (empirical)?
Two versions of the standard error are available for accompanying GEE estimates. The difference between them can be illustrated using the previously cited estimate, p = 0.42, of the parameter P. The "model-based" standard error is based on the estimated (exchangeable) correlation r = 0.45. This in turn implies the "effective sample size" of 3.39 (
w = 3 x 0.53 + 5 x 0.36 = 1.59 + 1.80 = 3.39) shown above and in the last footnote of figure 5. Thus, based on the binomial model,
SEmodel-based(p) = {p x (1 p)/
w}1/2 = {0.42 x 0.58/3.39}1/2 = 0.27.
The "empirical" or "robust" standard error uses the actual variations in the cluster-level statistics, that is, the p1 = 0/3 = 0 and p2 = 4/5 = 0.8, and the "effective sizes" of the subsamples
SEempirical(p) = {[1.592(0 0.42)2 + 1.802(0.8 0.42)2]/3.392}1/2 = 0.28.
Unless data are sparse, the empirical standard error is likely to be more trustworthy than the model-based one. Agreement between the model-based and empirical standard errors suggests that the assumed correlation structure is reasonable. However, the robust variance estimator, also known as the "sandwich" estimator, was developed for uncorrelated observations, and its theoretical behavior with correlated data has only recently received attention (19). Methods designed to improve on the poor performance in small samples (20) include bias-correction and explicit small-sample adjustments, that is, use of t rather than z (21). A second concern has been the case in which the cluster size itself is related to the outcome and so is "nonignorable." In such instances, within-cluster resampling, coupled with the use of a generalized linear model for uncorrelated data (13), provides more valid confidence intervals than GEE.
| APPLICATION |
|---|
|
|
|---|
Figure 6 showsfor the lower socioeconomic status group in table 1the various estimates of
|
The model-based and empirical standard errors agree to two decimal places in the case of
Figure 7 contrasts the Low and High socioeconomic status groups with respect to µ, mean height, and P, the proportion of children with z scores less than 1. We can estimate a difference by subtracting the specific estimates, and we can estimate its standard error from the rules for the variance of a difference between two independent estimates. Alternatively, the difference can be estimated as the coefficient of the indicator variable IHigh (1 if high socioeconomic status, 0 if not) in a regression model applied to the combined data. For height measured quantitatively, the intercept represents the mean of low socioeconomic status children, and the coefficient of IHigh represents the L-H difference in means.
|
GEE estimates of the proportions are shown in the right half of figure 7. The proportions are compared using various regression forms applied to the combined data. The slight discrepancy between the difference of the separately estimated group-specific proportions and the difference obtained directly from the regression model stems from the fact that the latter uses a common covariance rather than two separate covariances. The 11.0 percent standard error of the difference in proportions, calculated using the pooled covariances, and the (7.02 + 8.22)1/2 = 10.8 percent obtained from the two separate standard errors are nearly identical.
In the above examples, groups can be compared directly. However, to assess trends in responses over levels of one or more quantitative variables measured at a cluster level (here, household level), a regression approach is more practical. Since GEE analysis is carried out at the child level, it can also include covariates, such as illness histories, that differ from child to child within a household.
| DISCUSSION |
|---|
|
|
|---|
This orientation focused on correlated data arising from the relatedness of several individuals in the same cluster, rather than several "longitudinal" observations in the same individual. We chose examples that 1) could also be handled by classical methods and 2) were small enough to hand-calculate the weights induced by the correlations. These weights are used both to generate parameter estimates and to calculate standard errors. Although they are nuisance parameters, the correlations do provide for efficient estimates of the primary parameters and for accurate quantification of their precision.
The GEE approach differs in a fundamental conceptual way from the techniques included under the rubric of "random-effects," "multilevel," and "hierarchical" models (e.g., the MIXED and NLMIXED procedures in SAS, MLn (23, 24), or other software described in the paper by Burton et al. (5)). Besides the seeking of more efficient estimators of regression parameters, the main benefit of GEE is the production of reasonably accurate standard errors, hence confidence intervals with the correct coverage rates. The procedures in the other set of techniques explicitly model and estimate the between-cluster variation and incorporate this, and the residual variance, into standard errors. The GEE method does not explicitly model between-cluster variation; instead it focuses on and estimates its counterpart, the within-cluster similarity of the residuals, and then uses this estimated correlation to reestimate the regression parameters and to calculate standard errors. With GEE, the computational complexity is a function of the size of the largest cluster rather than of the number of clustersan advantage, and a source of reliable estimates, when there are many small clusters.
However, because the GEE approach does not contain explicit terms for the between-cluster variation, the resulting parameter estimates for the contrast of interest do not have the usual "keeping other factors constant" interpretation. To appreciate this, consider the (admittedly extreme) situation in table 2. If all N clusters are sufficiently large, one can fit an unconditional logistic regression model to the data. If clusters are small, one can avoid fitting one nuisance parameter per cluster (and the consequent bias in the estimated parameter of interest) and instead fit a more economical conditional logistic regression model, using each cluster as a "set." The appropriate logistic regression model "recovers" the common within-cluster ratio of 9, as does the nonregression Mantel-Haenszel approach. However, the GEE approach, with clusters identified as such, yields an odds ratio of only 5.4. The 5.4 contrasts the P1 for an individual selected randomly from the population with the P0 for another individual selected randomly from the population, that is, without "matching" on cluster. In addition, this "population averaged" measure, from the marginal model (5) used in the GEE approach, is specific to the mix of clusters studied. In contradistinction to this, the odds ratio of 9 contrasts the P1 for an individual with the P0 for another individual from the same cluster.
|
The subtleties of combining ratios, where the rules for "collapsibility" vary with the comparative measure (25), have long been recognized; indeed, the example of combining a 1 percent versus 5 percent contrast in one stratum (odds ratio ~ 5) and a 95 percent versus 99 percent contrast in the other (again, odds ratio ~ 5) was used by Mantel and Haenszel (26, p. 736). Gail et al. (27) used the even more extreme example with odds ratios of 9 (as in table 2) to show how a covariate omitted from a regression analysis can lead to attenuated estimates of what the authors call a "nonlinear" comparative parameter (such as the odds ratio and the hazard ratio), even ifas in table 2it is "balanced" across the compared levels of the factor.
The above extreme examples are quite hypothetical. In practice, with much less variation in P0 across clusters, the discrepancy is usually relatively minor. The discrepancy does not arise with absolute differences, since, with balanced sample sizes, the difference in an aggregate is the aggregate of the within-cluster differences. Table 2 confirms this, showing that the GEE approach, with the identity link, accurately recovers the common 40 percent "risk difference" within each cluster.
Unfortunately, as currently implemented in most software, the GEE approach cannot handle several levels of clustering/hierarchy, such as households selected from randomly selected villages that in turn were selected from selected counties. For binary responses, it is possible to use alternating logistic regression (28), an extension of GEE, implemented in S-PLUS, to model different correlations at different levels, but this procedure is not yet available in SPSS, Stata, and SAS implementations of GEE. Likewise, unlike multilevel models, the GEE approach cannot accommodate both cluster-specific intercepts and slopes in longitudinal data.
In our height example, several children within the household are measured cross-sectionally, that is, just once, each at a different age. Consider a different study, in which (unrelated) children are followed and their heights and covariates are measured at several different ages (times). In such longitudinal data, now with the child as the "cluster," unless the model includes at least a separate intercept for each child, the successive residual heights of a child will be correlated, with stronger correlations among residuals that are closer together in time. Autoregressive correlation structures are commonly used for longitudinal data. The main analytical challenges are accounting appropriately for missing data and dealing with observations spaced unevenly in time. The reader is referred to the work of Liang and Zeger and colleagues (1, 2, 4) for a treatment of the GEE analysis of quantitative longitudinal data.
| ACKNOWLEDGMENTS |
|---|
This research was supported by the Natural Sciences and Engineering Research Council of Canada (J. H.), grant CA 70269 from the US National Institutes of Health (A. N.), the Fonds de la Recherche en Santé du Québec (M. DeB. E.), and grants 5P01/DK45734-05 and R01 DA/KK11598-01 from the US National Institutes of Health (J. F.).
The authors are grateful to Rolf Heinmueller, Machelle Wilchesky, and several other students for comments on and reactions to the manuscript.
| APPENDIX |
|---|
|
|
|---|
|
| NOTES |
|---|
Correspondence to Dr. James A. Hanley, Department of Epidemiology and Biostatistics, Faculty of Medicine, McGill University, 1020 Pine Avenue West, Montreal, Quebec H3A 1A2, Canada (e-mail: james.hanley{at}mcgill.ca).
| REFERENCES |
|---|
|
|
|---|
- Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika 1986;73:1322.
[Abstract/Free Full Text] - Zeger SL, Liang KY. The analysis of discrete and continuous longitudinal data. Biometrics 1986;42:12130.[CrossRef][Web of Science][Medline]
- Wedderburn RW. Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika 1974;61:43947.
[Abstract/Free Full Text] - Diggle PJ, Liang KY, Zeger SL. Analysis of longitudinal data. Oxford, United Kingdom: Oxford University Press, 1994.
- Burton P, Gurrin L, Sly P. Extending the simple linear regression model to account for correlated responses: an introduction to generalized estimating equations and multi-level mixed modelling. Stat Med 1998;17:126191.[CrossRef][Web of Science][Medline]
- Horton NJ, Lipsitz SR. Review of software to fit generalized estimating equation regression models. Am Stat 1999;53:1609.
- Forrester JE, Scott ME, Bundy DA, et al. Predisposition of individuals and families in Mexico to heavy infection with Ascaris lumbricoides and Trichuris trichiura. Trans R Soc Trop Med Hyg 1990;84:2726.[CrossRef][Web of Science][Medline]
- Colton T. Statistics in medicine. Boston, MA: Little, Brown and Company, 1974.
- Rosner B, Milton RC. Significance testing for correlated binary outcome data. Biometrics1988;44:50512.[CrossRef][Web of Science][Medline]
- Donner A, Klar N. Methods for comparing event rates in intervention studies when the unit of allocation is a cluster. Am J Epidemiol 1994;140:27989.
[Abstract/Free Full Text] - Stansfield SK, Pierre-Louis M, Lerebours G, et al. Vitamin A supplementation and increased prevalence of childhood diarrhoea and acute respiratory infections. Lancet 1993;342:57882.[CrossRef][Web of Science][Medline]
- Gillman MD, Rifas-Shiman SL, Frazier AL, et al. Family dinner and diet quality among older children and adolescents. Arch Fam Med 2000;9:23540.
[Abstract/Free Full Text] - Hoffman EB, Sen PK, Weinberg CR. Within-cluster resampling. Biometrika 2001;61:43947.[CrossRef]
- Stigler SM. Least squares and the combination of observations. In: The history of statistics: the measurement of uncertainty before 1900. Cambridge, MA: Harvard University Press, 1986:1161.
- Nelder JA, Wedderburn RW. Generalized linear models. J R Stat Soc Ser A 1972;135:37084.[CrossRef]
- Armitage P, Berry G. Statistical methods in medical research. 3rd ed. Oxford, United Kingdom: Blackwell Scientific Publications, 1994.
- Numerical Algorithms Group Ltd. GLIM (Generalised Linear Interactive Modelling) software. Oxford, United Kingdom: Numerical Algorithms Group Ltd, 1995.
- Wacholder S. Binomial regression in GLIM: estimating risk ratios and risk differences. Am J Epidemiol 1986;123:17484.
[Abstract/Free Full Text] - Edwardes MD. Risk ratio and rate ratio estimation in case-cohort designs: hypertension and cardiovascular mortality. (Letter). Stat Med 1995;14:160910.[Web of Science][Medline]
- Breslow N. Tests of hypotheses in overdispersed Poisson regression and other quasi-likelihood models. J Am Stat Assoc 1990;85:56571.[CrossRef][Web of Science]
- Pan W, Wall M. Small-sample adjustments in using the sandwich variance estimator in generalized estimating equations. Stat Med 2002;21:142941.[CrossRef][Web of Science][Medline]
- Cochran WG. Sampling techniques. New York, NY: John Wiley and Sons, Inc, 1953:1247,2025.
- Goldstein H. Multilevel statistical models. 2nd ed. London, United Kingdom: Edward Arnold, 1995.
- Breslow N, Leroux B, Platt R. Approximate hierarchical modelling of discrete data in epidemiology. Stat Methods Med Res 1998;7:4962.
[Abstract/Free Full Text] - Boivin JF, Wacholder S. Conditions for confounding of the risk ratio and of the odds ratio. Am J Epidemiol 1985;121:1528.
[Abstract/Free Full Text] - Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 1959;22:71948.[Web of Science][Medline]
- Gail MH, Wieand S, Piantadosi S. Biased estimates of treatment effect in randomized experiments with non-linear regressions and omitted covariates. Biometrika 1984;71:43144.
[Abstract/Free Full Text] - Katz J, Carey VJ, Zeger SL, et al. Estimation of design effects and diarrhea clustering within households and villages. Am J Epidemiol 1993;138:9941006.
[Abstract/Free Full Text]
This article has been cited by other articles:
![]() |
D. L. Dahly, P. Gordon-Larsen, B. M. Popkin, J. S. Kaufman, and L. S. Adair Associations between Multiple Indicators of Socioeconomic Status and Obesity in Young Adult Filipinos Vary by Gender, Urbanicity, and Indicator Used J. Nutr., February 1, 2010; 140(2): 366 - 370. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. L. Halliday, O. C. Ukoumunne, H.W. G. Baker, S. Breheny, A. M. Jaques, C. Garrett, D. Healy, and D. Amor Increased risk of blastogenesis birth defects, arising in the first 4 weeks of pregnancy, after assisted reproductive technologies Hum. Reprod., January 1, 2010; 25(1): 59 - 65. [Abstract] [Full Text] [PDF] |
||||
![]() |
I. Shrier, R. J. Steele, J. Hanley, and B. Rich Analyses of Injury Count Data: Some Do's and Don'ts Am. J. Epidemiol., November 15, 2009; 170(10): 1307 - 1315. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. P. Luby, A. K. Halder, C. Tronchet, S. Akhter, A. Bhuiya, and R. B. Johnston Household Characteristics Associated with Handwashing with Soap in Rural Bangladesh Am J Trop Med Hyg, November 1, 2009; 81(5): 882 - 887. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Vakulin, S. D. Baulk, P. G. Catcheside, N. A. Antic, C. J. van den Heuvel, J. Dorrian, and R. D. McEvoy Effects of Alcohol and Sleep Restriction on Simulated Driving Performance in Untreated Patients With Obstructive Sleep Apnea Ann Intern Med, October 6, 2009; 151(7): 447 - 455. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Nakase-Richardson, S. A. Yablon, M. Sherer, T. G. Nick, and C. C. Evans Emergence from minimally conscious state: Insights from evaluation of posttraumatic confusion Neurology, October 6, 2009; 73(14): 1120 - 1126. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. S. Mercer, J. K. Freburger, S.-H. Chang, and J. L. Purser Step Test Scores Are Related to Measures of Activity and Participation in the First 6 Months After Stroke Physical Therapy, October 1, 2009; 89(10): 1061 - 1071. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Wake, L. A Baur, B. Gerner, K. Gibbons, L. Gold, J. Gunn, P. Levickis, Z. McCallum, G. Naughton, L. Sanci, et al. Outcomes and costs of primary care surveillance and intervention for overweight or obese children: the LEAP 2 randomised controlled trial BMJ, September 3, 2009; 339(sep03_2): b3308 - b3308. [Abstract] [Full Text] [PDF] |
||||
![]() |
M H W Wong, T A Dobbins, J Tseung, N Tran, C S Lee, C J O'Brien, J Clark, and B R Rose Oestrogen receptor {beta} expression in pleomorphic adenomas of the parotid gland J. Clin. Pathol., September 1, 2009; 62(9): 789 - 793. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Christensen, M. Pintilie, K. R. Evans, M. Lenarduzzi, C. Menard, C. N. Catton, E. P. Diamandis, and R. G. Bristow Longitudinal Cytokine Expression during IMRT for Prostate Cancer and Acute Treatment Toxicity Clin. Cancer Res., September 1, 2009; 15(17): 5576 - 5583. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. S. Dhamoon, Y. P. Moon, M. C. Paik, B. Boden-Albala, T. Rundek, R. L. Sacco, and M. S.V. Elkind Long-Term Functional Recovery After First Ischemic Stroke: The Northern Manhattan Study Stroke, August 1, 2009; 40(8): 2805 - 2811. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. E Flanagan, E. S Patterson, R. M Frankel, and B. N Doebbeling Evaluation of a Physician Informatics Tool to Improve Patient Handoffs JAMIA, July 1, 2009; 16(4): 509 - 515. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. P. Luby, M. Agboatwalla, A. Bowen, E. Kenah, Y. Sharker, and R. M. Hoekstra Difficulties in Maintaining Improved Handwashing Behavior, Karachi, Pakistan Am J Trop Med Hyg, July 1, 2009; 81(1): 140 - 145. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Anwaruddin, A. T. Askari, H. Saudye, L. Batizy, P. L. Houghtaling, M. Alamoudi, M. Militello, K. Muhammad, S. Kapadia, and S. G. Ellis Characterization of Post-Operative Risk Associated With Prior Drug-Eluting Stent Use J. Am. Coll. Cardiol. Intv., June 1, 2009; 2(6): 542 - 549. [Abstract] [Full Text] [PDF] |
||||
![]() |
M Suzuki, V D Thiem, H Yanai, T Matsubayashi, L-M Yoshida, L H Tho, T T Minh, D D Anh, P E Kilgore, and K Ariyoshi Association of environmental tobacco smoking exposure with an increased risk of hospital admissions for pneumonia in children under 5 years of age in Vietnam Thorax, June 1, 2009; 64(6): 484 - 489. [Abstract] [Full Text] [PDF] |
||||
![]() |
G Vizzeri, R N Weinreb, A O Gonzalez-Garcia, C Bowd, F A Medeiros, P A Sample, and L M Zangwill Agreement between spectral-domain and time-domain OCT for measuring RNFL thickness Br J Ophthalmol, June 1, 2009; 93(6): 775 - 781. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. M. Harley, B. A. English, and R. E. Ritzmann Characterization of obstacle negotiation behaviors in the cockroach, Blaberus discoidalis J. Exp. Biol., May 15, 2009; 212(10): 1463 - 1476. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Ahern, A. Hubbard, and S. Galea Estimating the Effects of Potential Public Health Interventions on Population Disease Burden: A Step-by-Step Illustration of Causal Inference Methods Am. J. Epidemiol., May 1, 2009; 169(9): 1140 - 1147. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. C. Koenen, A. E. Aiello, E. Bakshis, A. B. Amstadter, K. J. Ruggiero, R. Acierno, D. G. Kilpatrick, J. Gelernter, and S. Galea Modification of the Association Between Serotonin Transporter Genotype and Risk of Posttraumatic Stress Disorder in Adults by County-Level Social Environment Am. J. Epidemiol., March 15, 2009; 169(6): 704 - 711. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. P. Helzner, J. A. Luchsinger, N. Scarmeas, S. Cosentino, A. M. Brickman, M. M. Glymour, and Y. Stern Contribution of Vascular Risk Factors to the Progression in Alzheimer Disease Arch Neurol, March 1, 2009; 66(3): 343 - 348. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Moreau, J. Klastersky, A. Schwarzbold, F. Muanza, A. Georgala, M. Aoun, A. Loizidou, M. Barette, S. Costantini, M. Delmelle, et al. A general chemotherapy myelotoxicity score to predict febrile neutropenia in hematological malignancies Ann. Onc., March 1, 2009; 20(3): 513 - 519. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Treyvaud, V. A. Anderson, K. Howard, M. Bear, R. W. Hunt, L. W. Doyle, T. E. Inder, L. Woodward, and P. J. Anderson Parenting Behavior Is Associated With the Early Neurobehavioral Development of Very Preterm Children Pediatrics, February 1, 2009; 123(2): 555 - 561. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Reilly, M. Onslow, A. Packman, M. Wake, E. L. Bavin, M. Prior, P. Eadie, E. Cini, C. Bolzonello, and O. C. Ukoumunne Predicting Stuttering Onset by the Age of 3 Years: A Prospective, Community Cohort Study Pediatrics, January 1, 2009; 123(1): 270 - 277. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. L. Fortes, M. S. Allen, V. J. Lowe, K.-H. R. Shen, D. A. Wigle, S. D. Cassivi, F. C. Nichols, and C. Deschamps The sensitivity of 18F-fluorodeoxyglucose positron emission tomography in the evaluation of metastatic pulmonary nodules Eur. J. Cardiothorac. Surg., December 1, 2008; 34(6): 1223 - 1227. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Tabata, Z. Khalpey, L. H. Cohn, F. Y. Chen, R. M. Bolman III, and J. D. Rawn Effect of preoperative statins in patients without coronary artery disease who undergo cardiac surgery. J. Thorac. Cardiovasc. Surg., December 1, 2008; 136(6): 1510 - 1513. [Abstract] [Full Text] [PDF] |
||||
![]() |
L Giatti, S M Barreto, and C C. Cesar Household context and self-rated health: the effect of unemployment and informal work J Epidemiol Community Health, December 1, 2008; 62(12): 1079 - 1085. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Saposnik, T. Jeerakathil, D. Selchen, A. Baibergenova, V. Hachinski, M. K. Kapral, and for the Stroke Outcome Research Canada (SORCan) Wo Socioeconomic Status, Hospital Volume, and Stroke Fatality in Canada Stroke, December 1, 2008; 39(12): 3360 - 3366. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Borland, J. Balmford, N. Bishop, C. Segan, L. Piterman, L. McKay-Brown, C. Kirby, and C. Tasker In-practice management versus quitline referral for enhancing smoking cessation in general practice: a cluster randomized trial Fam. Pract., October 1, 2008; 25(5): 382 - 389. [Abstract] [Full Text] [PDF] |
||||
![]() |
I. H. Zuckerman, P. T. Ryder, L. Simoni-Wastila, T. Shaffer, M. Sato, L. Zhao, and B. Stuart Racial and Ethnic Disparities in the Treatment of Dementia Among Medicare Beneficiaries J Gerontol B Psychol Sci Soc Sci, September 1, 2008; 63(5): S328 - S333. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Hietala, C. Forsblom, P. Summanen, P.-H. Groop, and on behalf of the FinnDiane Study Group Heritability of Proliferative Diabetic Retinopathy Diabetes, August 1, 2008; 57(8): 2176 - 2180. [Abstract] [Full Text] [PDF] |
||||
![]() |
R Borland, G T Fong, H-H Yong, K M Cummings, D Hammond, B King, M Siahpush, A McNeill, G Hastings, R J O'Connor, et al. What happened to smokers' beliefs about light cigarettes when "light/mild" brand descriptors were banned in the UK? Findings from the International Tobacco Control (ITC) Four Country Survey Tob. Control, August 1, 2008; 17(4): 256 - 262. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. F. Brem, A. C. Floerke, J. A. Rapelyea, C. Teal, T. Kelly, and V. Mathur Breast-specific Gamma Imaging as an Adjunct Imaging Modality for the Diagnosis of Breast Cancer Radiology, June 1, 2008; 247(3): 651 - 657. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. W. Sesma, B. S. Slomine, R. Ding, M. L. McCarthy, and the Children's Health After Trauma (CHAT) Study Gr Executive Functioning in the First Year After Pediatric Traumatic Brain Injury Pediatrics, June 1, 2008; 121(6): e1686 - e1695. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Ahern, S. Galea, A. Hubbard, L. Midanik, and S. L. Syme "Culture of Drinking" and Individual Problems with Alcohol Use Am. J. Epidemiol., May 1, 2008; 167(9): 1041 - 1049. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. M. Coates and J. Herbert Endogenous steroids and financial risk taking on a London trading floor PNAS, April 22, 2008; 105(16): 6167 - 6172. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. M.A.J. Venmans, K. J. Gorter, E. Hak, and G. E.H.M. Rutten Short-Term Effects of an Educational Program on Health-Seeking Behavior for Infections in Patients With Type 2 Diabetes: A randomized controlled intervention trial in primary care Diabetes Care, March 1, 2008; 31(3): 402 - 407. [Abstract] [Full Text] [PDF] |
||||
![]() |
R Nakase-Richardson, S A Yablon, M Sherer, C C Evans, and T G Nick Serial yes/no reliability after traumatic brain injury: implications regarding the operational criteria for emergence from the minimally conscious state J. Neurol. Neurosurg. Psychiatry, February 1, 2008; 79(2): 216 - 218. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Herbeth, S. Gueguen, P. Leroy, G. Siest, and S. Visvikis-Siest The Lipoprotein Lipase Serine 447 Stop Polymorphism Is Associated With Altered Serum Carotenoid Concentrations in the Stanislas Family Study J. Am. Coll. Nutr., December 1, 2007; 26(6): 655 - 662. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Fotouhi, A. Etemadi, H. Hashemi, H. Zeraati, J. E Bailey-Wilson, and K. Mohammad Familial aggregation of myopia in the Tehran eye study: estimation of the sibling and parent offspring recurrence risk ratios Br J Ophthalmol, November 1, 2007; 91(11): 1440 - 1444. [Abstract] [Full Text] [PDF] |
||||
![]() |
I. J. Elkins, M. McGue, and W. G. Iacono Prospective Effects of Attention-Deficit/Hyperactivity Disorder, Conduct Disorder, and Sex on Adolescent Substance Use and Abuse Arch Gen Psychiatry, October 1, 2007; 64(10): 1145 - 1152. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Saposnik, A. Baibergenova, M. O'Donnell, M. D. Hill, M. K. Kapral, V. Hachinski, and On behalf of the Stroke Outcome Research Canada (S Hospital volume and stroke outcome: Does it matter? Neurology, September 11, 2007; 69(11): 1142 - 1151. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. J. Strawbridge, M. I. Wallhagen, and S. J. Shema Impact of Spouse Vision Impairment on Partner Health and Well-Being: A Longitudinal Analysis of Couples J Gerontol B Psychol Sci Soc Sci, September 1, 2007; 62(5): S315 - S322. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. R. Cole, M. A. Hernan, K. Anastos, B. D. Jamieson, and J. M. Robins Determining the Effect of Highly Active Antiretroviral Therapy on Changes in Human Immunodeficiency Virus Type 1 RNA Viral Load using a Marginal Structural Left-censored Mean Model Am. J. Epidemiol., July 15, 2007; 166(2): 219 - 227. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Koenigs and D. Tranel Irrational Economic Decision-Making after Ventromedial Prefrontal Damage: Evidence from the Ultimatum Game J. Neurosci., January 24, 2007; 27(4): 951 - 956. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Batra, C. L. Heike, R. C. Phillips, and N. S. Weiss Geographic and Occupational Risk Factors for Ventricular Septal Defects: Washington State, 1987-2003 Arch Pediatr Adolesc Med, January 1, 2007; 161(1): 89 - 95. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Ahern and S. Galea Social context and depression after a disaster: the role of income inequality. J Epidemiol Community Health, September 1, 2006; 60(9): 766 - 770. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Ahsan, Y. Chen, F. Parvez, L. Zablotska, M. Argos, I. Hussain, H. Momotaj, D. Levy, Z. Cheng, V. Slavkovich, et al. Arsenic Exposure from Drinking Water and Risk of Premalignant Skin Lesions in Bangladesh: Baseline Results from the Health Effects of Arsenic Longitudinal Study Am. J. Epidemiol., June 15, 2006; 163(12): 1138 - 1148. [Abstract] [Full Text] [PDF] |
||||
![]() |
J.P Webster, P.H.L Lamberton, C.A Donnelly, and E.F Torrey Parasites as causative agents of human affective disorders? The impact of anti-psychotic, mood-stabilizer and anti-parasite medication on Toxoplasma gondii's ability to alter host behaviour Proc R Soc B, April 22, 2006; 273(1589): 1023 - 1030. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. de Lauzon-Guillain, A. Basdevant, M. Romon, J. Karlsson, J.-M. Borys, M A. Charles, and The FLVS Study Group Is restrained eating a risk factor for weight gain in a general population? Am. J. Clinical Nutrition, January 1, 2006; 83(1): 132 - 138. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Galea, J. Ahern, S. Rudenstine, Z. Wallace, and D. Vlahov Urban built environment and depression: a multilevel analysis J Epidemiol Community Health, October 1, 2005; 59(10): 822 - 827. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. B Carlin, L. C Gurrin, J. A. Sterne, R. Morley, and T. Dwyer Regression models for twin studies: a critical review Int. J. Epidemiol., October 1, 2005; 34(5): 1089 - 1099. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Herbeth, E. Aubry, F. Fumeron, R. Aubert, F. Cailotto, G. Siest, and S. Visvikis-Siest Polymorphism of the 5-HT2A receptor gene and food intakes in children and adolescents: the Stanislas Family Study Am. J. Clinical Nutrition, August 1, 2005; 82(2): 467 - 470. [Abstract] [Full Text] [PDF] |
||||
![]() |
A.-S. Relova, L. D. Marrett, N. Klar, J. R. McLaughlin, F. D. Ashbury, D. Nishri, and B. Theis Predictors of Self-reported Confidence Ratings for Adult Recall of Early Life Sun Exposure Am. J. Epidemiol., July 15, 2005; 162(2): 183 - 192. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Geng, B. Kreiswirth, J. Burzynski, and N. W. Schluger Clinical and Radiographic Correlates of Primary and Reactivation Tuberculosis: A Molecular Epidemiology Study JAMA, June 8, 2005; 293(22): 2740 - 2745. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Maumus, B. Marie, G. Siest, and S. Visvikis-Siest A Prospective Study on the Prevalence of Metabolic Syndrome Among Healthy French Families: Two cardiovascular risk factors (HDL cholesterol and tumor necrosis factor-{alpha}) are revealed in the offspring of parents with metabolic syndrome Diabetes Care, March 1, 2005; 28(3): 675 - 682. [Abstract] [Full Text] [PDF] |
||||
![]() |
Blandine de Lauzon, M. Romon, V. Deschamps, L. Lafay, J.-M. Borys, J. Karlsson, P. Ducimetiere, and M. A. Charles The Three-Factor Eating Questionnaire-R18 Is Able to Distinguish among Different Eating Patterns in a General Population J. Nutr., September 1, 2004; 134(9): 2372 - 2380. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. P van Alem, R. H Vrenken, R. de Vos, J. G P Tijssen, and R. W Koster Use of automated external defibrillator by first responders in out of hospital cardiac arrest: prospective controlled trial BMJ, December 6, 2003; 327(7427): 1312. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. W Kissane, M. McKenzie, D. P McKenzie, A. Forbes, I. O'Neill, and S. Bloch Psychosocial morbidity associated with patterns of family functioning in palliative care: baseline data from the Family Focused Grief Therapy controlled trial Palliative Medicine, September 1, 2003; 17(6): 527 - 537. [Abstract] [PDF] |
||||
![]() |
J Merlo Multilevel analytical approaches in social epidemiology: measures of health variation compared with traditional measures of association J Epidemiol Community Health, August 1, 2003; 57(8): 550 - 552. [Full Text] [PDF] |
||||
![]() |
J. H. Godbold RE: "STATISTICAL ANALYSIS OF CORRELATED DATA USING GENERALIZED ESTIMATING EQUATIONS: AN ORIENTATION" Am. J. Epidemiol., August 1, 2003; 158(3): 289 - 289. [Full Text] [PDF] |
||||
![]() |
G. Zou RE: "STATISTICAL ANALYSIS OF CORRELATED DATA USING GENERALIZED ESTIMATING EQUATIONS: AN ORIENTATION" Am. J. Epidemiol., August 1, 2003; 158(3): 289 - 289. [Full Text] [PDF] |
||||
![]() |
J. A. Hanley THE FIRST AUTHOR REPLIES Am. J. Epidemiol., August 1, 2003; 158(3): 289 - 290. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||














































