Skip Navigation


American Journal of Epidemiology Advance Access originally published online on May 25, 2007
American Journal of Epidemiology 2007 166(2):137-150; doi:10.1093/aje/kwm086
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
166/2/137    most recent
kwm086v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (17)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Kim, J. J.
Right arrow Articles by Goldie, S. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kim, J. J.
Right arrow Articles by Goldie, S. J.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

American Journal of Epidemiology © The Author 2007. Published by the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

PRACTICE OF EPIDEMIOLOGY

Multiparameter Calibration of a Natural History Model of Cervical Cancer

Jane J. Kim1, Karen M. Kuntz1, Natasha K. Stout1, Salaheddin Mahmud2, Luisa L. Villa3, Eduardo L. Franco2 and Sue J. Goldie1

1 Program in Health Decision Science, Department of Health Policy and Management, Harvard School of Public Health, Boston, MA
2 Division of Cancer Epidemiology, McGill University, Montreal, Quebec, Canada
3 Ludwig Institute for Cancer Research, São Paulo, Brazil

Correspondence to Dr. Jane J. Kim, Program in Health Decision Science, Harvard School of Public Health, 718 Huntington Avenue, 2nd Floor, Boston, MA 02115 (e-mail: jkim{at}hsph.harvard.edu).

Received for publication September 12, 2006. Accepted for publication January 25, 2007.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
The objective of this study was to develop a comprehensive natural history model of human papillomavirus (HPV) and cervical cancer using a two-step approach to model calibration. In the first step, the authors utilized primary epidemiologic data from a longitudinal study of women in Brazil and identified a plausible range for each input parameter that produced model output within the 95% confidence intervals of the data. In the second step, they performed a simultaneous search over all input parameters to identify parameter sets that produced output consistent with data from multiple sources. A goodness-of-fit score was computed for 555,000 unique parameter sets using a likelihood-based approach, and a sample of good-fitting parameter sets was used in the model to illustrate the advantage of the calibration approach by projecting a range of benefits associated with cervical cancer prevention policies. The calibrated model had reasonable fit to the data in terms of duration and prevalence of HPV infection for high-risk types, prevalence of precancerous lesions, and incidence of cancer. The authors found that leveraging primary data from longitudinal studies provides unique opportunities for model parameterization of the unobservable nature of HPV infection and its role in the development of cervical cancer.

calibration; computer simulation; human papillomavirus 16; human papillomavirus 18; natural history; papillomavirus vaccines; uterine cervical neoplasms

Abbreviations: CIN, cervical intraepithelial neoplasia; HPV, human papillomavirus; HSIL, high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesion


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Over the past several years, our understanding of the natural history of cervical cancer, and specifically the role of human papillomavirus (HPV) as its central causal agent, has increased dramatically (1–3). With the availability of reliable assays to detect high-risk, oncogenic types of HPV, and the recent licensure of a vaccine targeting HPV types 16 and 18 (4)—together responsible for as much as 70 percent of invasive cancer—many important questions have emerged about how screening guidelines should be modified to capitalize on synergies between screening and vaccination and, for low-resource settings in particular, how investments in vaccination will compare with adoption of newly proposed screening strategies applied two to three times per lifetime (5).

No single empirical study can evaluate all possible strategies to inform these complex policy questions. Integrating the best biologic, epidemiologic, and economic data, computer-based mathematical models can assist in early decision making, can identify those factors most likely to influence outcomes, and can provide insight into the potential cost-effectiveness of different strategies (6). Disease-specific simulation models used for this purpose are constructed to simulate the course of disease in individuals and project the overall health impact of the disease in a population. In doing so, they must consider the underlying, and oftentimes unobserved, course of disease. As a general rule, complete data for all input parameters are not available, and calibration to epidemiologic data is necessary to estimate the unknown probabilities (7).

As the nature of the policy questions for cervical cancer control becomes more complex, sophisticated and detailed models of cervical carcinogenesis are also needed. Unfortunately, the number of unobserved natural history parameters in such a model quickly multiplies. At the same time, epidemiologic data on incidence and prevalence of type-specific HPV, cervical intraepithelial neoplasia (CIN) and cancer, and distribution of HPV types within CIN and cancer are increasingly available (8–16). Proper use of these data requires new approaches for model calibration that take into account the uncertainty of both input parameters and calibration targets but constrain model output to be consistent with the best epidemiologic data. This study describes the development of a comprehensive natural history model of cervical cancer using an iterative multistep approach to calibration.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 APPENDIX
 References
 
Model
We developed a stochastic Monte Carlo simulation model of cervical carcinogenesis, in which disease progression in an individual patient is characterized as a sequence of monthly transitions between health states (figure 1). Health states in the model, descriptive of each patient's underlying true health, are defined to include HPV infection status, grade of CIN, and stage of invasive cancer; they are further stratified by HPV type categorized as 1) high-risk type 16; 2) high-risk type 18; 3) other high-risk types, including 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68, 73, and 82; and 4) low-risk types, including 6, 11, 26, 32, 34, 40, 42, 44, 53, 54, 55, 57, 61, 62, 64, 67, 69, 70, 71, 72, 81, 83, and 84. The latter hierarchical classification was used because of its strong empirical value in stratifying risk predictions. Note that categories 3 and 4 include HPV types that belong to different species and may vary in their oncogenic potential.


Figure 1
View larger version (13K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 1. Schematic of the natural history model of cervical cancer. Health states were defined using four general categories of cervical health (normal, infection with human papillomavirus (HPV), grade of cervical intraepithelial neoplasia (CIN), and stage of invasive cancer). The model simulates the natural history of cervical carcinogenesis by using a sequence of monthly transitions among these health states. Women face an age-dependent risk of acquiring HPV, and their cervical disease can progress or regress subject to probabilities conditional on their HPV status. Not shown are unique health states, which were defined to distinguish women with previously abnormal screening tests, prior treatment for CIN, and detected cervical cancer (through symptoms or screening). Women may die from cervical cancer or other causes. Details of the model are provided in the Materials and Methods section.

 
Individual women enter the model prior to sexual debut and are simulated one at a time using a series of monthly transitions between health states. The probabilities governing these transitions depend on age, HPV type, and type-specific natural immunity, as well as a woman's history of prior infection, CIN, and patterns of screening. Natural immunity was modeled as a reduction in future type-specific infection after initial infection and clearance. Each month, death can occur from noncervical cancer causes from all health states, or from cervical cancer after its onset. Each patient's lifetime clinical course is tracked from start age until death, with a tally maintained of all clinical events and accrued costs. The model also distinguishes the true underlying health status of each woman (e.g., CIN grade 2,3) from the reported results of a screening test (e.g., high-grade squamous intraepithelial lesion (HSIL)) to simulate data observed in a clinical study. In general, 10 million women are simulated, one at a time, to provide stable estimates of long-term outcomes for each strategy.

A number of assumptions were necessary for the model. First, consistent with the latest scientific evidence (1–3), we assumed that invasive cancer could not occur in the absence of infection with a high-risk HPV type. Second, we assumed that our probabilities of age-related HPV incidence serve as a proxy for sexual risk by age. Third, because the natural history implications of simultaneous infections of multiple types are uncertain and occur in less than 10 percent of our study population (17), we elected to classify women in this study with single infections according to the same hierarchical risk scale used in epidemiologic studies. For example, a woman with HPV type 16 and type 6 would be classified according to the more relevant high-risk HPV type 16.

Calibration using primary data
The initial estimates for the natural history model parameters (table 1) were based on data from the published literature (18–43) and have been documented elsewhere (44, 45). For our initial calibration step, we used primary data from the Ludwig-McGill cohort study, a longitudinal study of the natural history of HPV infection and cervical neoplasia in a population of low-income women in Sao Paulo, Brazil, one of the highest risk areas worldwide for cervical cancer (8). From 1993 to 1997, the study enrolled over 2,400 women between the ages of 18 and 60 years and scheduled repeat follow-up visits at 4 months, 8 months, 12 months, and every 6 months thereafter for up to 8 years. At each visit, a cervical specimen was collected for both cytology testing and HPV DNA testing using polymerase chain reaction. Abnormal cytology results were categorized as low-grade squamous intraepithelial lesion (LSIL) or HSIL, and HPV status was categorized as no HPV, low-risk HPV, or high-risk HPV.


View this table:
[in this window]
[in a new window]

  		TABLE 1. Model parameters: baseline values and calibration ranges

 
We calibrated to three outcomes estimated from the Ludwig-McGill study: 1) mean duration (in months) of HPV infection, defined as time from incident low-risk or high-risk HPV infection to either clearance or development of LSIL or HSIL (46, 47); 2) hazard ratios for the association between HPV status at enrollment and incident cytologic abnormalities within 6 years of follow-up (48); and 3) hazard ratios for the association between time-varying HPV status and first occurrence of cytologic abnormalities within 3 years of follow-up (48). We simulated a population similar to the study population; specifically, when a healthy female enters the model at age 12 years, the study enrollment age and screening schedule are drawn from a nonparametric distribution of enrollment age and visit schedule from the Ludwig-McGill study. The woman progresses through the natural history model and, when a screening month arrives, receives both a cytology test and HPV DNA test, and the results are recorded. We repeated this simulation for 100,000 women and produced a data set of observations containing age, visit month, HPV status, and cytology result, similar to the Ludwig-McGill data set. We used the same statistical algorithm as the one in the Ludwig-McGill study to calculate hazard ratios (details provided in the Appendix) (48).

We simulated the Ludwig-McGill study population using baseline parameters derived from the literature as used in a previous model (44, 45) and compared the model output with the observed data. Over numerous iterations, we systematically adjusted parameter values over a broad range to explore the independent influence of these parameters on model output and to visually improve the fit of the model to the three outcomes from the Ludwig-McGill study. We assumed that the age patterns of probabilities governing the transition between health states were fixed (Appendix figures 1–4) but that the magnitude of these probabilities varied; as such, we applied multipliers to the parameters and varied the multipliers over a broad range. We further explored the uncertainty that exists regarding the timing and degree of type-specific natural immunity.

From this first calibration step, we generated plausible ranges for all input parameters with which the model projected estimates of HPV duration and hazard ratios that fell within or near the 95 percent confidence intervals for the equivalent observed statistics in the Ludwig-McGill study. We also visually checked each simulation to ensure that model predictions of age-specific prevalence of HPV and CIN and age-specific cancer incidence followed trends similar to those reported in the literature.

Calibration using a likelihood-based approach
While the goal of our first calibration exercise was to explore the influence of individual parameters using original data from the Ludwig-McGill study, our second phase of calibration was intended to allow all parameters to vary simultaneously and to capitalize on the full range of available data. In doing so, we sought to identify sets of parameter values that, when used in the model, produced outcomes that were consistent with epidemiologic data from multiple sources. As in the first calibration step, we applied multipliers to the parameters and sampled over a range of multiplier values (table 1). Multiple simulations of the natural history model were conducted. For a single simulation, one value for each parameter was randomly selected from a uniform distribution over the identified range. In total, simulations were conducted with 555,000 uniquely sampled parameter sets.

Model outcomes using each parameter set were compared with multiple epidemiologic targets (table 2), including the outcomes from the Ludwig-McGill study; age-specific prevalence of low-risk and high-risk HPV, CIN 1, and CIN 2,3; type distributions of HPV among women with CIN and cancer; and age-specific incidence of invasive cervical cancer. Using data from meta-analyses and the published literature (10–16, 46–58), we specified likelihood functions for each target, assuming that each followed an independent normal distribution. Although prevalence data should in theory follow binomial distributions, we found that the normal approximation to the binomial was appropriate for the majority of prevalence estimates and therefore assumed normal distributions in order to remain consistent across all epidemiologic targets used for calibration. For the majority of targets, the normal distributions were constructed around the mean point estimate obtained from a single study. For the cancer incidence targets, which were based on cancer registry data from multiple sites and years, we estimated separate age-specific cancer incidence curves for each site and year, and we selected the lowest and highest curves to represent the lower and upper bounds of the data.


View this table:
[in this window]
[in a new window]

  		TABLE 2. Calibration targets: parameters of normal distributions*

 
Goodness of fit for each parameter set was computed by summing the log-likelihood of each model outcome. We used the likelihood ratio test to identify a good-fitting subset, comprising those parameter sets that did not produce a significantly worse fit than the best-fitting parameter set (using an alpha level of 5 percent). In sensitivity analysis, we considered alternative scoring algorithms for assessing goodness of fit; for example, using a binary score for each target output, we computed a composite score indicating the number of model outcomes that fell outside of the 95 percent confidence intervals of the corresponding target data. Furthermore, we explored a hierarchical scoring system in which we prioritized model fit to select targets based on the richness of available data or the importance of the output. With each of these scoring approaches, we used alternative thresholds for defining "good-fitting" sets (e.g., top 5, top 50, and top 100 scoring sets).

For illustrative purposes, we used a subset of 50 good-fitting runs from the likelihood-based calibration step to estimate the mean and range of reductions in lifetime cancer risk associated with two-visit HPV DNA testing strategies in Brazil. Details on the two-visit HPV DNA testing strategy have been provided elsewhere (5, 59) but are briefly summarized here. Women undergo initial screening during the first visit and then return for results during a second visit. Screen-positive women undergo visual inspection to determine whether they are suitable for cryosurgery treatment; women deemed ineligible are referred for diagnostic testing (e.g., colposcopy/biopsy) and, if necessary, receive treatment for precancerous lesions, depending upon lesion size and type. Results were generated for different screening intervals ranging from once per lifetime to every 3 years.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 APPENDIX
 References
 
Model fit to primary data
Table 3 compares the mean duration of type-specific HPV infection from the Ludwig-McGill study (46, 47) with model output and shows the projected lifetime cancer risk from the model using adjusted natural history parameter values. Using baseline parameter values from the literature, which mostly reflect average probabilities rather than HPV type-specific probabilities, the model produced mean durations of both low-risk and high-risk HPV infections that were much longer than observed in the Ludwig-McGill study. To better approximate the empirical data, we performed simulations in which we applied multipliers greater than 1 in increments of 0.5 to rates of HPV incidence and clearance. A fourfold increase in average incidence and clearance was required to produce mean estimates of duration that were within the 95 percent confidence intervals for the high-risk types. However, in doing so, the estimates of duration for low-risk HPV remained below the 95 percent confidence interval until progression and regression of CIN were also adjusted. After these changes were implemented, the model estimates of HPV duration approximated the empirical data but the corresponding lifetime risk of cancer estimated for the study population was 10.8 percent, higher than what would be expected for Sao Paulo, Brazil.


View this table:
[in this window]
[in a new window]

  		TABLE 3. Influence of natural history parameter values on duration of HPV* infection and projected lifetime risk of cancer

 
After varying remaining model parameters and finding they had minimal influence on model fit, we systematically explored the impact of alternative assumptions such as proportion of women with HPV infection progressing directly to CIN 2,3; heterogeneity among women with different HPV types in regression from CIN 2,3; and degree of natural immunity to type-specific infection. Only when we introduced type-specific natural immunity consistently above 50 percent for high-risk HPV types did model fit improve, simultaneously fitting estimates of duration from the Ludwig-McGill study and producing an estimate for lifetime cancer risk of 3.1 percent, which is consistent with the annual incidence rate in Sao Paulo (15, 16).

Figure 2 shows the hazard ratio estimates for the associations between HPV status at enrollment (upper panel) and at a given visit (bottom panel) and incident LSIL or HSIL from the Ludwig-McGill study (48) compared with the model output. In general, we found that most of the scenarios we examined fell within the 95 percent confidence intervals of the observed data, with some projecting estimates closer to the mean values than others.


Figure 2
View larger version (21K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 2. Hazard ratios for the association between human papillomavirus (HPV) status at enrollment and incident cytologic abnormalities within 6 years (top panel), and HPV status at a given visit and incident cytologic abnormalities within 3 years (bottom panel). White bars, empirical data from the Ludwig-McGill cohort study (48); bars with vertical lines, model projections using the initial parameter values; bars with dots, analysis in which HPV incidence and clearances were multiplied by a factor of four; bars with diagonal lines, analysis in which multipliers were allowed to vary across the noncancer transition parameters; and black bars, analysis in which the assumption of immunity was introduced (consistently greater than 50% for high-risk (HR) HPV, and in this particular scenario it was 90%). Error bars represent the 95% confidence intervals of the Ludwig-McGill study estimates. Top panel, hazard ratios for associations of low-risk (LR) and HR HPV at enrollment with incident low-grade squamous intraepithelial lesion (LSIL) and incident high-grade squamous intraepithelial lesion (HSIL) within 6 years. Bottom panel, hazard ratios for associations of LR and HR HPV at a given visit with first squamous intraepithelial lesion (SIL) event and first HSIL event within 3 years.

 
Model fit using a likelihood-based approach
Using the likelihood ratio test, we identified 16,818 parameter sets (of the 555,000 sampled sets) that were statistically indistinguishable from the set that achieved the best fit. Figures 3, 4, and 5 show the model output compared with the 95 percent confidence intervals for three of the calibration target categories for a selection of 50 good-fitting sets. Although projected prevalences of low-risk HPV at younger and older ages were low, the model had reasonable fit to the data in terms of duration of HPV infection for both low-risk and high-risk types; prevalence of high-risk HPV infection, CIN 1, and CIN 2,3; and incidence of cancer. When we used either the binary or the hierarchical scoring algorithms, the top 100 scoring sets were included in the 16,818 good-fitting sets identified by the likelihood-based scoring approach. The parameter values for a sample of three good-fitting sets are presented in Appendix table 1.


Figure 3
View larger version (16K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 3. Model output of age-specific prevalence of high-risk (HR) human papillomavirus (HPV). Dotted lines, model output for 50 sampled good-fitting sets. Solid lines, 95% confidence intervals of the target data at each age group (10, 11, 50–53).

 

Figure 4
View larger version (9K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 4. Type distribution of high-risk 16 (HR-16), high-risk 18 (HR-18), and high-risk other (HR-other) human papillomavirus infection among women with cervical intraepithelial neoplasia (CIN) 1, CIN 2,3, and invasive cancer. Dots, model output for 50 sampled good-fitting sets. Solid lines, 95% confidence intervals of the target data (12, 14, 56–58).

 

Figure 5
View larger version (21K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 5. Age-specific incidence of cervical cancer per 100,000 women in Brazil. Dotted lines, model output for 50 sampled good-fitting sets. Solid lines, 95% confidence intervals of age-specific incidence data from the International Agency for Research on Cancer (15, 16).

 
Illustrative example of model output
To depict the nature of output using the calibrated model, figure 6 displays the mean reduction in lifetime risk of cervical cancer associated with two-visit HPV DNA testing using the sample of 50 good-fitting sets. The error bars represent the minimum and maximum reductions achieved by the 50 good-fitting sets. Also plotted is the output from a previous deterministic model using a single parameter set that was derived solely from the published literature and was not subject to calibration. The reductions in cancer risk projected from the deterministic model were similar to the mean reductions from the calibrated good-fitting sets when screening was frequent, but results slightly diverged when screening was less frequent.


Figure 6
View larger version (11K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 6. Comparison of reduction in lifetime risk of cervical cancer for a two-visit human papillomavirus (HPV) DNA testing strategy. White bars, predicted reductions using 50 sampled good-fitting sets. Black bars, predicted reductions using baseline precalibrated parameter set. Error bars, minimum and maximum reductions achieved by the 50 good-fitting sets. Each strategy was compared with a strategy of no screening.

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
APPENDIX
 References
 
Despite substantial progress in our understanding of cervical carcinogenesis and the development of advanced preventive measures, cervical cancer continues to be a leading cause of cancer death among women worldwide (60). Motivated to address policy questions that require consideration of new screening technology, individualized screening protocols that depend on prior history, and the availability of type-specific vaccination, we developed a stochastic model that distinguishes HPV types, reflects heterogeneity among women, and accommodates more complex cervical cancer control strategies. The details and complexities of the new model introduce many challenges with respect to the increased number of unobserved parameters, however. Our objective in this study was to describe the iterative process in which we first utilized primary data for model development to identify key influential parameters and then used likelihood-based methods to identify sets of parameter values that produced model outcomes consistent with epidemiologic data from multiple sources.

In our initial calibration step, capitalizing on primary data from the Ludwig-McGill cohort study, we identified a set of model parameters describing the underlying transitions among HPV and CIN states that calibrate well to estimates of HPV duration and hazard ratios for the association of low- and high-risk HPV status at enrollment with detection of LSIL and HSIL. In general, we found that good fits were possible only with increased transition rates for HPV infection and clearance, increased transition rates for CIN progression and regression, and the introduction of type-specific natural immunity following HPV infection and clearance. Specifically, only when we assumed that natural immunity exceeded 50 percent for high-risk HPV types was a reasonable fit to the Ludwig-McGill data achieved. This finding was consistent with data from an empirical study that observed reduced risk of subsequent HPV-16 infection (61). Overall, this exploratory exercise illustrated the drawbacks of underestimating type-specific probabilities when using only average rates reported in the literature, and it highlighted the importance of including natural immunity in the parameterization of HPV incidence.

In our second phase of model development, we used a more formal likelihood-based calibration approach that enabled us to search simultaneously over multiple parameters to identify parameter sets that fit well to a wide range of epidemiologic data from multiple independent sources. Using 50 good-fitting parameter sets, we then provided an example of a model-based analysis of two-visit HPV DNA testing across different screening frequencies to demonstrate how the output from this model compares with that of a simpler deterministic model that relies on average transition rates that are generally not type specific. As expected, we found that the average outcomes associated with interventions aimed at screening and treatment for CIN were similar between the two models; however, our empirically calibrated model enabled us to report a range of cost-effectiveness ratios, thereby providing a better representation of the uncertainty of our results for decision makers. As we begin to address questions that incorporate interventions directed at specific HPV types, such as vaccination against HPV-16/-18 infection, we expect that the average results produced by the two models will diverge.

Although other calibration approaches have been described in the literature (62–68), no universal approach is sufficient for all models, nor is there consensus on the "best" calibration approach. In fact, the choice of calibration method by an analyst depends on many factors, including the uncertainty with respect to the natural history of the disease, the nature of available data, and the model structure complexity required to assess the interventions of interest. An alternate approach to model calibration—Bayesian methods—capitalizes on prior information on input parameters and generates a joint posterior distribution over all model parameters based on both the prior information and the likelihood of the different sets of parameter values conditional on the observed data (67, 68). Despite its theoretical appeal, however, the Bayesian approach requires definition of meaningful priors, identification of likelihood functions for the set of observed data, and complex computations. The difficulties in using a Bayesian approach have generally outweighed the benefits in the calibration of disease-specific models, both deterministic and stochastic. Our analytic approach to calibration was selected after carefully considering the strengths and limitations—both theoretical and practical—of Bayesian and other calibration methods. We believe the approach we described here will appeal to epidemiologists because it uses common measures of effect produced in cross-sectional, case-control, and cohort studies as part of the approach to model calibration.

There are limitations to our approach, including the model structure and data inputs. First, although HPV is a sexually transmitted infection, we do not explicitly reflect transmission of HPV in the population based on sexual behavior but instead use data on age-related HPV incidence as a proxy. However, our model has the capability to be linked to an independent transmission model in which the probability of an individual acquiring an infection is dependent on the detailed sexual contact patterns of that individual and the distribution of the infection within the population at a given time. There are also inherent limitations and errors in the epidemiologic data used as inputs and calibration targets; for example, study subjects may not be representative of the general population, which may lead to under- or overestimation of the effects associated with interventions evaluated at the population level. In addition, although longitudinal data are critical in simulation modeling, biases may result from loss to follow-up and from changes in measurement instruments over time (e.g., HPV DNA assay) (69, 70).

Our calibration methods also have several limitations. With multiple parameters being varied simultaneously, the search space was extensive; even with more than half a million parameter sets sampled, it is difficult to know whether the space was searched comprehensively. Although more efficient optimization algorithms (71) can be used to guide the parameter search more systematically, we chose the less efficient random search to maximize coverage of the space and to avoid identification of a local rather than the global maximum (67). Furthermore, to our knowledge, no benchmark exists for how "good" the model fit must be to ensure adequate representation of a disease process. In this analysis, we used a likelihood ratio test to identify parameter sets statistically similar to the best-fitting parameter set. However, this approach relies on the assumption that the best set itself achieves an acceptable fit to the empirical data. After exploring alternative scoring algorithms and acceptance criteria, we consistently identified the same best-fitting sets, which gave us confidence about the reliability of our likelihood-based scoring approach.

From our iterative, multistep calibration approach, we constructed a revised natural history model of cervical cancer that is better equipped than our previous models to capture many important aspects of cervical cancer prevention policies. First, because of our calibration methods, which enable us to identify and utilize multiple good-fitting parameter sets, the model has the capability of provide a range of results, allowing more complete representation of the impact of uncertainty on policy results (72). Second, we integrated new epidemiologic data on the complex natural history of HPV infections, enabling us to evaluate new screening technologies that detect high-risk types of HPV and preventive vaccines against HPV-16 and -18 that are imminent (73). Finally, and perhaps most valuable, this iterative process of model development using empirical calibration permits us to leverage the wealth of new information about HPV anticipated in coming years and to tailor our modeling efforts to different countries. By providing a more solid foundation for evaluating cervical cancer control policies, we aim to address the most important policy questions related to screening and vaccination in a timely manner.


   APPENDIX
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
References
 
Hazard ratios were calculated by using traditional Cox proportional hazards models in Stata, version 7 software (Stata Corporation, College Station, Texas) (48). Four different hazard ratios were calculated at each follow-up month to determine the association of HPV status at enrollment and incident LSIL or HSIL within the specified period of follow-up:

  1. Hazard ratios of low-risk HPV at enrollment and LSIL within visit x:

    Formula

  2. Hazard ratios of low-risk HPV at enrollment and HSIL within visit x:

    Formula

  3. Hazard ratios of high-risk HPV at enrollment and LSIL within visit x:

    Formula

  4. Hazard ratios of high-risk HPV at enrollment and HSIL within visit x:

    Formula

To reflect changing HPV status during the study, a time-dependent Cox proportional hazards model was used to estimate the hazard ratios for the association of HPV status at a given visit and first occurrence of LSIL or HSIL over a 3-year follow-up period. Women with prevalent lesions at enrollment were excluded from the analysis, and women who developed high-grade lesions were censored from further analysis.


View this table:
[in this window]
[in a new window]

  		APPENDIX TABLE 1. Sample of good-fitting multiplier sets from calibration using the likelihood-based approach*

 


Figure 1
View larger version (13K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		APPENDIX FIGURE 1. Age-specific baseline probabilities of human papillomavirus (HPV), prior to calibration. Baseline probabilities for type-specific HPV incidence, estimated from a longitudinal cohort study of women from Colombia and data from the published literature (18–24). LR, low risk; HR, high risk.

 


Figure 2
View larger version (9K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		APPENDIX FIGURE 2. Age-specific baseline probabilities of cervical intraepithelial neoplasia (CIN) 1, prior to calibration. Baseline probabilities for progression of human papillomavirus (HPV) (low-risk (LR) and high-risk (HR) types) to CIN 1, based on data from the published literature (23–27).

 


Figure 3
View larger version (9K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		APPENDIX FIGURE 3. Age-specific baseline probabilities of cervical intraepithelial neoplasia (CIN) 2,3, prior to calibration. Baseline probabilities for progression of CIN 1 to CIN 2,3, based on data from the published literature (28–36). LR, low risk; HR, high risk.

 


Figure 4
View larger version (8K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		APPENDIX FIGURE 4. Age-specific baseline probabilities of cervical cancer, prior to calibration. Baseline probabilities for progression of cervical intraepithelial neoplasia 2,3 to local cervical cancer, based on data from the Surveillance, Epidemiology, and End Results program registry (23, 24, 37). HR, high risk; HPV, human papillomavirus.

 

   ACKNOWLEDGMENTS
 
Funded by grants from the Bill and Melinda Gates Foundation (30505), National Cancer Institute (CA93435, CA70269), and Canadian Institutes of Health Research (MA-13647 and MOP-49396) and by intramural support from the Ludwig Institute for Cancer Research.

The authors gratefully acknowledge the contributions of the Global HPV Vaccine Policy Model Team, including Joshua A. Salomon, Jeremy D. Goldhaber-Fiebert, Katie E. Kobus, Jesse Ortendahl, Meredith O'Shea, Steven Sweet, Bethany Andres-Beck, Nicole Gastineau Campos, and Mireia Diaz-Sanchis.

Conflict of interest: none declared.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 

  1. Franco EL, Rohan TE, Villa LL. Epidemiologic evidence and human papillomavirus infection as a necessary cause of cervical cancer. J Natl Cancer Inst (1999) 91:506–11.[Free Full Text]
  2. Walboomers JM, Jacobs MV, Manos MM, et al. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol (1999) 189:12–19.[CrossRef][Web of Science][Medline]
  3. Bosch FX, Lorincz A, Munoz N, et al. The causal relation between human papillomavirus and cervical cancer. J Clin Pathol (2002) 55:244–65.[Abstract/Free Full Text]
  4. Food and Drug Administration. FDA licenses new vaccine for prevention of cervical cancer and other diseases in females caused by human papillomavirus. Press release, 2006. (http://www.fda.gov/bbs/topics/NEWS/2006/NEW01385.html).
  5. Goldie SJ, Gaffikin L, Goldhaber-Fiebert JD, et al. Cost-effectiveness of cervical cancer screening in five developing countries. N Engl J Med (2005) 353:2158–68.[Abstract/Free Full Text]
  6. Goldie SJ, Goldhaber-Fiebert JD, Garnett GP. Chapter 18: public health policy for cervical cancer prevention: the role of decision science, economic evaluation, and mathematical modeling. Vaccine (2006) 24(suppl 3):S155–63.[CrossRef][Medline]
  7. Garnett GP, Kim JJ, French K, et al. Chapter 21: modelling the impact of HPV vaccines on cervical cancer and screening programmes. Vaccine (2006) 24(suppl 3):S178–86.[CrossRef][Medline]
  8. Franco E, Villa L, Rohan T, et al. Design and methods of the Ludwig-McGill longitudinal study of the natural history of human papillomavirus infection and cervical neoplasia in Brazil. Ludwig-McGill Study Group. Rev Panam Salud Publica (1999) 6:223–33.[Medline]
  9. Herrero R, Hildesheim A, Bratti C, et al. Population-based study of human papillomavirus infection and cervical neoplasia in rural Costa Rica. J Natl Cancer Inst (2000) 92:464–74.[Abstract/Free Full Text]
  10. Molano M, Posso H, Weiderpass E, et al. Prevalence and determinants of HPV infection among Colombian women with normal cytology. Br J Cancer (2002) 87:324–33.[CrossRef][Web of Science][Medline]
  11. Clifford GM, Gallus S, Herrero R, et al. Worldwide distribution of human papillomavirus types in cytologically normal women in the International Agency for Research on Cancer HPV prevalence surveys: a pooled analysis. Lancet (2005) 366:991–8.[CrossRef][Web of Science][Medline]
  12. Clifford GM, Rana RK, Franceschi S, et al. Human papillomavirus genotype distribution in low-grade cervical lesions: comparison by geographic region and with cervical cancer. Cancer Epidemiol Biomarkers Prev (2005) 14:1157–64.[Abstract/Free Full Text]
  13. Clifford GM, Smith JS, Aguado T, et al. Comparison of HPV type distribution in high-grade cervical lesions and cervical cancer: a meta-analysis. Br J Cancer (2003) 89:101–5.[CrossRef][Web of Science][Medline]
  14. Clifford GM, Smith JS, Plummer M, et al. Human papillomavirus types in invasive cervical cancer worldwide: a meta-analysis. Br J Cancer (2003) 88:63–73.[CrossRef][Web of Science][Medline]
  15. Waterhouse J, Muir C, Correa P, et al, eds. Cancer incidence in five continents. (1976) Vol III. Lyon, France: International Agency for Research on Cancer. (IARC scientific publication no. 15).
  16. Parkin DM, Whelan SL, Ferlay J, et al, eds. Cancer incidence in five continents. (2002) Vol VIII. Lyon, France: International Agency for Research on Cancer. (IARC scientific publication no. 155).
  17. Trottier H, Mahmud S, Costa MC, et al. Human papillomavirus infections with multiple types and risk of cervical neoplasia. Cancer Epidemiol Biomarkers Prev (2006) 15:1274–80.[Abstract/Free Full Text]
  18. Munoz N, Mendez F, Posso H, et al. Incidence, duration, and determinants of cervical human papillomavirus infection in a cohort of Colombian women with normal cytological results. J Infect Dis (2004) 190:2077–87.[CrossRef][Web of Science][Medline]
  19. Hildesheim A, Schiffman MH, Gravitt PE, et al. Persistence of type-specific human papillomavirus infection among cytologically normal women. J Infect Dis (1994) 169:235–40.[Web of Science][Medline]
  20. Ho GY, Bierman R, Beardsley L, et al. Natural history of cervicovaginal papillomavirus infection in young women. N Engl J Med (1998) 338:423–8.[Abstract/Free Full Text]
  21. Liu T, Soong SJ, Alvarez RD, et al. A longitudinal analysis of human papillomavirus 16 infection, nutritional status, and cervical dysplasia progression. Cancer Epidemiol Biomarkers Prev (1995) 4:373–80.[Abstract]
  22. Sun XW, Kuhn L, Ellerbrock TV, et al. Human papillomavirus infection in women infected with the human immunodeficiency virus. N Engl J Med (1997) 337:1343–9.[Abstract/Free Full Text]
  23. McCrory D, Mather D, Bastian L, et al. Evaluation of cervical cytology. (1999) Rockville, MD: Agency for Health Care Policy and Research. (Evidence report/technology assessment no. 5).
  24. Myers ER, McCrory DC, Nanda K, et al. Mathematical model for the natural history of human papillomavirus infection and cervical carcinogenesis. Am J Epidemiol (2000) 151:1158–71.[Abstract/Free Full Text]
  25. Londesborough P, Ho L, Terry G, et al. Human papillomavirus genotype as a predictor of persistence and development of high-grade lesions in women with minor cervical abnormalities. Int J Cancer (1996) 69:364–8.[CrossRef][Web of Science][Medline]
  26. Syrjanen K, Kataja V, Yliskoski M, et al. Natural history of cervical human papillomavirus lesions does not substantiate the biologic relevance of the Bethesda System. Obstet Gynecol (1992) 79:675–82.[Web of Science][Medline]
  27. Ho GY, Burk RD, Klein S, et al. Persistent genital human papillomavirus infection as a risk factor for persistent cervical dysplasia. J Natl Cancer Inst (1995) 87:1365–71.[Abstract/Free Full Text]
  28. Ostor AG. Natural history of cervical intraepithelial neoplasia: a critical review. Int J Gynecol Pathol (1993) 12:186–92.[Web of Science][Medline]
  29. Melnikow J, Nuovo J, Willan AR, et al. Natural history of cervical squamous intraepithelial lesions: a meta-analysis. Obstet Gynecol (1998) 92:727–35.[CrossRef][Web of Science][Medline]
  30. Ho GY, Kadish AS, Burk RD, et al. HPV 16 and cigarette smoking as risk factors for high-grade cervical intra-epithelial neoplasia. Int J Cancer (1998) 78:281–5.[CrossRef][Web of Science][Medline]
  31. Hopman EH, Rozendaal L, Voorhorst FJ, et al. High risk human papillomavirus in women with normal cervical cytology prior to the development of abnormal cytology and colposcopy. BJOG (2000) 107:600–4.[CrossRef][Medline]
  32. Rozendaal L, Walboomers JM, van der Linden JC, et al. PCR-based high-risk HPV test in cervical cancer screening gives objective risk assessment of women with cytomorphologically normal cervical smears. Int J Cancer (1996) 68:766–9.[CrossRef][Web of Science][Medline]
  33. Ellerbrock TV, Chiasson MA, Bush TJ, et al. Incidence of cervical squamous intraepithelial lesions in HIV-infected women. JAMA (2000) 283:1031–7.[Abstract/Free Full Text]
  34. Koutsky LA, Holmes KK, Critchlow CW, et al. A cohort study of the risk of cervical intraepithelial neoplasia grade 2 or 3 in relation to papillomavirus infection. N Engl J Med (1992) 327:1272–8.[Abstract]
  35. Nobbenhuis MA, Walboomers JM, Helmerhorst TJ, et al. Relation of human papillomavirus status to cervical lesions and consequences for cervical-cancer screening: a prospective study. Lancet (1999) 354:20–5.[CrossRef][Web of Science][Medline]
  36. Remmink AJ, Walboomers JM, Helmerhorst TJ, et al. The presence of persistent high-risk HPV genotypes in dysplastic cervical lesions is associated with progressive disease: natural history up to 36 months. Int J Cancer (1995) 61:306–11.[Web of Science][Medline]
  37. Ries LA, Eisner MP, Kosary CL, et al, eds. SEER cancer statistics review, 1975–2001. (2004) Bethesda, MD: National Cancer Institute. (http://seer.cancer.gov/csr/1975_2001/2004).
  38. Fahey MT, Irwig L, Macaskill P. Meta-analysis of Pap test accuracy. Am J Epidemiol (1995) 141:680–9.[Abstract/Free Full Text]
  39. Nanda K, McCrory DC, Myers ER, et al. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann Intern Med (2000) 132:810–19.[Abstract/Free Full Text]
  40. Wright TC Jr, Schiffman M, Solomon D, et al. Interim guidance for the use of human papillomavirus DNA testing as an adjunct to cervical cytology for screening. Obstet Gynecol (2004) 103:304–9.[Web of Science][Medline]
  41. Bergeron C, Jeannel D, Poveda J, et al. Human papillomavirus testing in women with mild cytologic atypia. Obstet Gynecol (2000) 95:821–7.[CrossRef][Web of Science][Medline]
  42. Ferris DG, Wright TC Jr, Litaker MS, et al. Triage of women with ASCUS and LSIL on Pap smear reports: management by repeat Pap smear, HPV DNA testing, or colposcopy? J Fam Pract (1998) 46:125–34.[Web of Science][Medline]
  43. Ferris DG, Wright TC Jr, Litaker MS, et al. Comparison of two tests for detecting carcinogenic HPV in women with Papanicolaou smear reports of ASCUS and LSIL. J Fam Pract (1998) 46:136–41.[Web of Science][Medline]
  44. Kim JJ, Wright TC, Goldie SJ. Cost-effectiveness of alternative triage strategies for atypical squamous cells of undetermined significance. JAMA (2002) 287:2382–90.[Abstract/Free Full Text]
  45. Goldie SJ, Kim JJ, Wright TC. Cost-effectiveness of human papillomavirus DNA testing for cervical cancer screening in women aged 30 years or more. Obstet Gynecol (2004) 103:619–31.[Web of Science][Medline]
  46. Franco EL, Villa LL, Sobrinho JP, et al. Epidemiology of acquisition and clearance of cervical human papillomavirus infection in women from a high-risk area for cervical cancer. J Infect Dis (1999) 180:1415–23.[CrossRef][Web of Science][Medline]
  47. Schlecht NF, Platt RW, Duarte-Franco E, et al. Human papillomavirus infection and time to progression and regression of cervical intraepithelial neoplasia. J Natl Cancer Inst (2003) 95:1336–43.[Abstract/Free Full Text]
  48. Schlecht NF, Platt RW, Negassa A, et al. Modeling the time dependence of the association between human papillomavirus infection and cervical cancer precursor lesions. Am J Epidemiol (2003) 158:878–86.[Abstract/Free Full Text]
  49. Sadeghi SB, Sadeghi A, Robboy SJ. Prevalence of dysplasia and cancer of the cervix in a nationwide, planned parenthood population. Cancer (1988) 61:2359–61.[CrossRef][Web of Science][Medline]
  50. Kotloff KL, Wasserman SS, Russ K, et al. Detection of genital human papillomavirus and associated cytological abnormalities among college women. Sex Transm Dis (1998) 25:243–50.[Web of Science][Medline]
  51. Bauer HM, Hildesheim A, Schiffman MH, et al. Determinants of genital human papillomavirus infection in low-risk women in Portland, Oregon. Sex Transm Dis (1993) 20:274–8.[Web of Science][Medline]
  52. Ley C, Bauer HM, Reingold A, et al. Determinants of genital human papillomavirus infection in young women. J Natl Cancer Inst (1991) 83:997–1003.[Abstract/Free Full Text]
  53. Melkert PW, Hopman E, van den Brule AJ, et al. Prevalence of HPV in cytomorphologically normal cervical smears, as determined by the polymerase chain reaction, is age-dependent. Int J Cancer (1993) 53:919–23.[Web of Science][Medline]
  54. Mount SL, Papillo JL. A study of 10,296 pediatric and adolescent Papanicolaou smear diagnoses in northern New England. Pediatrics (1999) 103:539–45.[Abstract/Free Full Text]
  55. Lawson HW, Lee NC, Thames SF, et al. Cervical cancer screening among low-income women: results of a national screening program, 1991 –1995. Obstet Gynecol (1998) 92:745–52.[CrossRef][Web of Science][Medline]
  56. Lorenzato F, Ho L, Terry G, et al. The use of human papillomavirus typing in detection of cervical neoplasia in Recife (Brazil). Int J Gynecol Cancer (2000) 10:143–50.[CrossRef][Web of Science][Medline]
  57. Rabelo-Santos SH, Zeferino L, Villa LL, et al. Human papillomavirus prevalence among women with cervical intraepithelial neoplasia III and invasive cervical cancer from Goiania, Brazil. Mem Inst Oswaldo Cruz (2003) 98:181–4.[Web of Science][Medline]
  58. Eluf-Neto J, Booth M, Munoz N, et al. Human papillomavirus and invasive cervical cancer in Brazil. Br J Cancer (1994) 69:114–19.[Web of Science][Medline]
  59. Goldie SJ, Kuhn L, Denny L, et al. Policy analysis of cervical cancer screening strategies in low-resource settings: clinical benefits and cost-effectiveness. JAMA (2001) 285:3107–15.[Abstract/Free Full Text]
  60. Yang BH, Bray FI, Parkin DM, et al. Cervical cancer as a priority for prevention in different world regions: an evaluation using years of life lost. Int J Cancer (2004) 109:418–24.[CrossRef][Web of Science][Medline]
  61. Ho GY, Studentsov Y, Hall CB, et al. Risk factors for subsequent cervicovaginal human papillomavirus (HPV) infection and the protective role of antibodies to HPV-16 virus-like particles. J Infect Dis (2002) 186:737–42.[CrossRef][Web of Science][Medline]
  62. Salomon JA, Weinstein MC, Hammitt JK, et al. Empirically calibrated model of hepatitis C virus infection in the United States. Am J Epidemiol (2002) 156:761–73.[Abstract/Free Full Text]
  63. Cancre N, Tall A, Rogier C, et al. Bayesian analysis of an epidemiologic model of Plasmodium falciparum malaria infection in Ndiop, Senegal. Am J Epidemiol (2000) 152:760–70.[Abstract/Free Full Text]
  64. Craig BA, Fryback DG, Klein R, et al. A Bayesian approach to modelling the natural history of a chronic condition from observations with intervention. Stat Med (1999) 18:1355–71.[CrossRef][Web of Science][Medline]
  65. Stout NK, Fryback DG, Rosenberg MA, et al. Construction, verification, and calibration of a discrete event simulation model of breast cancer epidemiology in Wisconsin. (Abstract). October. Baltimore, Maryland: Presented at the 24th Annual Meeting for the Society for Medical Decision Making. 19–23, 2002.
  66. Knudsen AB, Weinstein MC, Kuntz KM. Modeling unobserved processes: a likelihood-based approach to model calibration. (Abstract). Chicago, Illinois: Presented at the 25th Annual Meeting for the Society for Medical Decision Making. October 18–22, 2003.
  67. Kennedy MC, O'Hagan A. Bayesian calibration of computer models. J R Stat Soc (B) (2001) 63:425–64.[CrossRef]
  68. Andradottir S, Bier VM. Applying Bayesian ideas in simulation. Simul Pract Theory (2000) 8:253–80.[CrossRef]
  69. Franco EL. Measurement errors in epidemiological studies of human papillomavirus and cervical cancer. IARC Sci Publ (1992) 181–97.
  70. Franco EL. Statistical issues in human papillomavirus testing and screening. Clin Lab Med (2000) 20:345–67.[Web of Science][Medline]
  71. Nelder JA, Mead R. A simplex method for function minimization. Comput J (1965) 7:303–33.
  72. Hunink MGM, Glasziou P, Siegel JE, et al. Decision making in health and medicine: integrating evidence and values. Chapter 11. (2001) New York, NY: Cambridge University Press.
  73. Franco EL, Cuzick J, Hildesheim A, et al. Chapter 20: issues in planning cervical cancer screening in the era of HPV vaccination. Vaccine (2006) 24(suppl 3):S171–7.[CrossRef][Medline]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 ANN INTERN MEDHome page
J. J. Kim, J. Ortendahl, and S. J. Goldie
Cost-Effectiveness of Human Papillomavirus Vaccination and Cervical Cancer Screening in Women Older Than 30 Years in the United States
Ann Intern Med, October 20, 2009; 151(8): 538 - 545.
[Abstract] [Full Text] [PDF]

Home page
 BMJHome page
J. J Kim and S. J Goldie
Cost effectiveness analysis of including boys in a human papillomavirus vaccination programme in the United States
BMJ, October 8, 2009; 339(oct08_2): b3884 - b3884.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
J.-F. Shi, J. L. Belinson, F.-H. Zhao, R. G. Pretorius, J. Li, J.-F. Ma, F. Chen, W. Xiang, Q.-J. Pan, X. Zhang, et al.
Human Papillomavirus Testing for Cervical Cancer Screening: Results From a 6-Year Prospective Study in Rural China
Am. J. Epidemiol., September 15, 2009; 170(6): 708 - 716.
[Abstract] [Full Text] [PDF]

Home page
 Proc. Natl. Acad. Sci. USAHome page
S. Basu, G. B. Chapman, and A. P. Galvani
Integrating epidemiology, psychology, and economics to achieve HPV vaccination targets
PNAS, December 2, 2008; 105(48): 19018 - 19023.
[Abstract] [Full Text] [PDF]

Home page
 NEJMHome page
J. J. Kim and S. J. Goldie
Health and Economic Implications of HPV Vaccination in the United States
N. Engl. J. Med., August 21, 2008; 359(8): 821 - 832.
[Abstract] [Full Text] [PDF]

Home page
 Cancer Epidemiol. Biomarkers Prev.Home page
J. M. Yeh, K. M. Kuntz, M. Ezzati, C. Hur, C. Y. Kong, and S. J. Goldie
Development of an Empirically Calibrated Model of Gastric Cancer in Two High-Risk Countries
Cancer Epidemiol. Biomarkers Prev., May 1, 2008; 17(5): 1179 - 1187.
[Abstract] [Full Text] [PDF]

Home page
 Am J EpidemiolHome page
S. Basu and A. P. Galvani
RE: "MULTIPARAMETER CALIBRATION OF A NATURAL HISTORY MODEL OF CERVICAL CANCER"
Am. J. Epidemiol., October 15, 2007; 166(8): 983 - 983.
[Full Text] [PDF]

Home page
 Am J EpidemiolHome page
J. J. Kim, E. L. Franco, N. K. Stout, and S. J. Goldie
THE AUTHORS REPLY
Am. J. Epidemiol., October 15, 2007; 166(8): 983 - 984.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
166/2/137    most recent
kwm086v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (17)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Kim, J. J.
Right arrow Articles by Goldie, S. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kim, J. J.
Right arrow Articles by Goldie, S. J.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?