American Journal of Epidemiology Advance Access published online on November 26, 2008
American Journal of Epidemiology, doi:10.1093/aje/kwn342
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Original Contribution |
Is Relative Mortality of Type 2 Diabetes Mellitus Decreasing?
Correspondence to Dr. Martin Gulliford, Department of Public Health Sciences, Kings College London, Capital House, 42 Weston Street, London SE1 3QD, United Kingdom (e-mail: martin.gulliford{at}kcl.ac.uk).
Received for publication June 12, 2008. Accepted for publication September 26, 2008.
| ABSTRACT |
|---|
|
|
|---|
There is concern that the increasing prevalence of type 2 diabetes may diminish improving trends in life expectancy. This study aimed to determine whether the mortality of type 2 diabetes, relative to mortality in the general population, is remaining constant. The study included a cohort of 48,556 subjects with type 2 diabetes first diagnosed between 1996 and 2006, drawn from 197 family practices in the United Kingdom General Practice Research Database. There were 6,630 deaths observed. Expected mortality was estimated from United Kingdom life tables. Relative mortality was modeled using Poisson regression. In men with diabetes, from 1996 to 2006, the age-standardized all-cause mortality rate decreased by 0.82/1,000 per year (95% confidence interval (CI): 0.36, 1.27) and by 0.49 (95% CI: 0.29, 0.68) in women with diabetes. After adjustment for age, sex, and diabetes duration, there was a consistent decrease in relative mortality during the period of study. Relative mortality for subjects diagnosed in 1996 was 13% (95% CI: 2, 25) higher than that in 2001; for subjects diagnosed in 2006, relative mortality was 26% (95% CI: 8, 40) lower than that in 2001. Relative mortality of type 2 diabetes appears to be decreasing in men and women in the United Kingdom.
diabetes mellitus, type 2; mortality
Abbreviations: CI, confidence interval; GPRD, General Practice Research Database; MHRA, Medicines and Healthcare Products Regulatory Agency
| INTRODUCTION |
|---|
|
|
|---|
The prevalence of type 2 diabetes is increasing worldwide. In the United States, the age-adjusted prevalence of known diabetes mellitus increased from 2.8% in 1980 to 5.3% in 2005 (1). In comparison with the general population, subjects with type 2 diabetes have considerably increased mortality (2, 3), and there is concern that the increasing prevalence of diabetes and associated conditions may lead to a reduction or reversal in the present improving trends in life expectancy in higher income countries (4–6). One recent analysis suggests that a reversal of mortality trends may already be taking place in poor communities (7).
Several studies show that mortality in type 2 diabetic persons is now declining, with a greater decline in men than in women (8, 9). The significance of this decrease is unclear, because mortality rates have also been declining in the general population (10, 11). The concept of relative mortality is used to compare mortality in a given study population with mortality in the general population; relative mortality is the ratio of mortality rates in study and reference populations. In their comprehensive review up to 1995, Geiss et al. (3) concluded that the age-adjusted mortality of persons with diabetes may be up to twice that of persons who do not have diabetes. The aim of the present study was to determine whether the relative mortality associated with type 2 diabetes changed between 1996 and 2006. We estimated the relative mortality of a United Kingdom population-based cohort of subjects with newly diagnosed type 2 diabetes by comparing observed mortality with the mortality expected for people of the same age and sex in the United Kingdom general population.
| MATERIALS AND METHODS |
|---|
|
|
|---|
Design and subjects
We analyzed data from the United Kingdom General Practice Research Database (GPRD), a large database including electronic patient records for approximately 5% of United Kingdom family practices. In the United Kingdom, there is population-based provision of primary care, with 98% of the population being registered with a family practice. The quality of GPRD data has been shown to be good (12). The study protocol was approved by the Independent Scientific Advisory Committee for Medicines and Healthcare Products Regulatory Agency (MHRA) Database Research (protocol number 07-019). Data from this study have been reported elsewhere (13).
The present longitudinal study included all 197 family practices, with a registered population of approximately 1.63 million, that contributed data to GPRD continuously between January 1, 1995, and December 31, 2006 (13). We selected the population aged 30 years or over, because most subjects diagnosed at younger ages have type 1 diabetes. Prevalent cases of diabetes were identified from the electronic clinical record if they were ever diagnosed with diabetes or prescribed oral hypoglycemic drugs or insulin. A diagnosis of diabetes was identified if a medical diagnostic code for diabetes was documented in the electronic medical record. Details of the codes used are available from the authors. The date of diagnosis was identified as the earlier of the first medical diagnosis for diabetes or the first medical prescription for hypoglycemic drugs. Only subjects newly diagnosed between January 1, 1996, and December 31, 2006, were included in the cohort, and subjects entered the cohort on the date of diagnosis. All subjects therefore had a minimum of 12 months record before the date of diagnosis of diabetes. The end date for the analysis was March 30, 2007, the date of the latest data collection. Incident cases that were ever diagnosed with type 1 diabetes or prescribed insulin within 6 months of diagnosis were excluded as cases of type 1 diabetes. Dates of death were ascertained from the electronic medical record. A previous study has shown that mortality rates can be reliably estimated from GPRD records (14).
Analysis
Age-standardized mortality rates for men and women in the United Kingdom general population were estimated by using Interim Life Tables for the United Kingdom. This was done because mortality rates are not published for the United Kingdom but are only issued for the constituent countries (England and Wales, Scotland, and Northern Ireland). United Kingdom life tables are published annually based on deaths and population estimates for successive 3-year periods (14). At the time of analysis, the most recent life table covered the period 2004–2006, and these data were used for both 2005 and 2006. Expected deaths were estimated by applying the probability of mortality in the interval to the standard population. For comparison, age standardization was also performed by using age- and sex-specific mortality rates for England and Wales giving similar results. Age standardization for the United Kingdom general population was implemented for the population aged 30 years or over in 1-year age groups. The European standard population is published in 5-year age groups. We assumed that the standard population was equally divided among years of age within 5-year age categories. Age standardization for the diabetes sample was implemented by using the age groups 30–44, 45–54, 55–64, 65–74, 75–84, and
85 years, and rates were estimated by year of death. These were chosen to facilitate presentation and because the data for deaths were spread across 11 years of study. Year of death was estimated as year of diagnosis plus number of completed years since diabetes diagnosis. The European standard population was used for reference, and confidence intervals for age-standardized rates were estimated by using the "normal" approximation to the binomial distribution.
In order to estimate relative mortality in diabetes, we compared the observed survival of subjects in the diabetic sample with the expected survival of subjects of the same sex and year of age, for the same calendar year, in the United Kingdom general population. The United Kingdom Interim Life Tables were used for reference (15). The "strs" command in STATA, version 9, software (StataCorp LP, College Station, Texas) was used to estimate expected deaths and person-years at risk by age group, sex, year of diagnosis, and duration of diabetes (16, 17). Relative mortality was estimated as the ratio of observed and expected mortality; this is equivalent to the standardized mortality ratio. Confidence intervals for relative mortality were estimated based on the Poisson distribution. A Poisson regression model for observed deaths was fitted to the grouped data, with expected deaths as offset, to estimate the associations of relative mortality with age, sex, duration of diabetes, and year of diagnosis. Tests of significance are 2 sided throughout.
| RESULTS |
|---|
|
|
|---|
There were 48,579 subjects with newly diagnosed type 2 diabetes between 1996 and 2006, and there were 6,635 deaths during the period from January 1, 1996, to March 30, 2007. After omitting 23 cases with missing age or discrepant values, we analyzed data for 48,556 cases and 6,630 deaths (13).
Figure 1 shows mortality rates for men and women aged 30 years or over. Data are presented for the diabetes cohort and for the United Kingdom general population, standardized to the distribution of the European standard population. Data for 1996 were omitted as the 58 deaths occurring in this year were considered to be too few to provide stable estimates for age-standardized rates. Mortality rates declined in the general population, but mortality rates observed in the diabetes cohort declined more rapidly, consistent with higher relative mortality from diabetes in 1997 than in 2006. Mortality rates were higher in men than women, and the decline in diabetes mortality generally appeared to be more consistent in men than women. However, relative mortality was higher in women. In diabetic subjects, the annual decrease in the age-standardized mortality rate per 1,000 was 0.82 (95% confidence interval (CI): 0.36, 1.27) in men and 0.49 (95% CI: 0.29, 0.68) in women. In the United Kingdom general population, annual decreases in age-standardized mortality were smaller, being 0.35 (95% CI: 0.33, 0.37) in men and 0.18 (95% CI: 0.17, 0.20) in women. Based on comparison of age-standardized rates, relative mortality for men declined from 1.38 in 1997 to 1.27 in 2006, and in women it declined from 1.62 in 1997 to 1.44 in 2006.
|
Table 1 presents data for deaths, person-years, and relative mortality by year of diagnosis and duration of diabetes for men and women. The estimate for relative mortality represents the ratio of observed deaths to deaths expected if general population mortality rates were applied to a sample of the same age and sex. Approximate expected values are given by observed deaths divided by relative mortality. Relative mortality was highest in the first 2 years after diabetes diagnosis and declined as the duration of diabetes increased. Relative mortality declined with increasing year of diagnosis, and this was generally consistent for each duration of diabetes. Note that relative mortality data are standardized to the age distribution of the diabetes sample that has a different age structure from that of the European standard population used for the rates presented in Figure 1.
|
Table 2 shows the results of the regression model for relative mortality. The model leads to the estimation of a ratio comparing relative mortality in different groups. This model showed that the relative mortality associated with type 2 diabetes was higher for women than men. Relative mortality from diabetes was greatest in the youngest age groups and declined with age. Relative mortality also declined with increasing duration of diabetes. After adjustment for age group, sex, and duration of diabetes, it was clear that relative mortality declined with increasing year of diagnosis.
|
Relative mortality for subjects diagnosed in 1996 was 13% (95% CI: 2, 25) higher than for subjects diagnosed in 2001; for subjects diagnosed in 2006, relative mortality was 26% (95% CI: 8, 40) lower than for those diagnosed in 2001. A test of interaction showed no evidence that the effect of year of diagnosis differed between men and women (P = 0.385).
Table 3 presents data for relative mortality by age at diagnosis and duration of diabetes for men and women combined. Relative mortality was highest in the youngest age group and decreased with age. Relative mortality was high close to diagnosis of diabetes and declined as the duration of diabetes increased. In the youngest age group, relative mortality later increased as the duration of diabetes increased, but this was not observed in older age groups. A test for interaction between age at diagnosis and duration of diabetes gave P < 0.001. Data for durations of diabetes of 10–11 years were not presented as there were too few cases. Durations of diabetes longer than 11 years were not represented in the sample.
|
| DISCUSSION |
|---|
|
|
|---|
Interpretation
The present results suggest that relative mortality in type 2 diabetes may be declining over time in the United Kingdom. Although decreasing relative mortality has some potential to moderate the anticipated impacts of diabetes on life expectancy, the outcome will also depend on trends in the prevalence of diabetes at different ages (4–6). There are several possible explanations for the decline in relative mortality of type 2 diabetes. First, long-term declining trends in mortality from coronary heart disease and stroke (18–20), as well as decreasing exposure to risk factors including cigarette smoking, may have had a differentially important impact on the mortality of diabetic subjects. However, some studies suggest that diabetic populations may not have benefited from these favorable trends (21). Second, active case finding and screening for type 2 diabetes have been encouraged in recent years (22). The impact is uncertain, but this may have facilitated the rapid increase in clinically diagnosed diabetes with declining age at diagnosis of diabetes (23). Diabetes may now be ascertained at an earlier stage, before complications are established, leading to an improved early prognosis consistent with lead-time bias. Third, results of large clinical trials have provided evidence to support medical intervention to control blood glucose (24, 25), blood pressure (26), and cholesterol (27, 28) with the aim of reducing complications and mortality. Combinations of interventions may be particularly effective at reducing mortality (28, 29). Improving standards of chronic illness care have contributed to increasing uptake of these interventions. There may also have been increased use of statins and antihypertensive therapy before the diagnosis of diabetes, contributing to improved prognosis (13). All of these developments have the potential to modify the excess mortality associated with type 2 diabetes.
Higher relative mortality from type 2 diabetes at younger ages has been documented in previous studies (3) and was confirmed here. This suggests that the relative impact of diabetes is greatest at ages when the underlying risk of mortality in the general population is lowest. In older adults, in whom mortality risk is high, the relative impact of diabetes is smaller. Our data suggest that subjects who survive to be newly diagnosed with diabetes in old age may represent a particular group with lower mortality risk.
Higher relative mortality close to the time of diagnosis is consistent with other studies that show a considerable excess of ischemic heart disease and cerebrovascular disease events around the time of diagnosis of diabetes (30). It will therefore be relevant to explore whether the clinical context of diabetes diagnoses is changing over time.
Strengths and limitations
This study has the strength of a large sample drawn from a population-based database. In the United Kingdom, there is population-based provision of primary care with universal eligibility and access to primary care. Previous studies of mortality in the GPRD suggest that mortality rates are similar to those observed in vital registrations (14). By utilizing a cohort design including incident cases of diabetes, we were able to estimate separate associations of duration of diabetes and year of diagnosis of diabetes with mortality, and this is generally not feasible if prevalent cases are utilized. However, the cohort was followed for a maximum of 11 years, with shorter follow-up for more recently diagnosed cases, and we have not evaluated trends for longer durations of diabetes. Estimation of relative mortality requires that the condition of interest and deaths from the condition of interest have negligible frequency in the reference population, and this assumption may be less satisfactory for type 2 diabetes than for rare conditions, leading to underestimation of relative mortality. However, if the prevalence of diabetes is 5% and relative mortality is 2, then this underestimation would be about 10%. A different approach would be to sample control subjects from GPRD who were never diagnosed with diabetes. However, subjects selected because they were never diagnosed with diabetes have considerably lower comorbidity than subjects who later develop diabetes, and the former may provide a biased comparison group (31). Identification of diabetes cases was based on clinical diagnoses. There may have been subjects included that later reverted to normal glucose tolerance; however, subjects with subclinical diabetes were not included. We did not have recorded information about socioeconomic position or ethnicity. It is possible that favorable trends in relative mortality are not shared equally by all groups in the population, but we were not able to evaluate this possibility. We also note that this study generally had lower power to evaluate relative mortality in the youngest or oldest age groups or at longer durations of diabetes.
Analysis of cause-specific mortality would yield additional information that would aid in interpretation of the present observations. However, causes of death are not immediately available within GPRD. Linkage of vital registration data would therefore be informative.
Our regression-based estimates of relative mortality appear to be lower than those obtained through direct comparison of the age-standardized rates shown in Figure 1. However, as noted by Greenland and Rothman (32, p. 262), the 2 estimates are not directly comparable because they are standardized to different age distributions. The diabetes sample has an older age distribution, and this gives greater weight to age groups with lower relative mortality. For the same reason, it is difficult to compare estimates of relative mortality between studies because relative mortality is highly dependent on age, being more than 3 for those diagnosed at ages 30–44 years but close to 1 for subjects diagnosed at the age of 85 years. Increased relative mortality close to the time of diagnosis is consistent with the increased incidence of cardiovascular events at this time (30). Because the diagnosis of diabetes may be less secure in some cases, we repeated the analysis after excluding 399 women initially diagnosed with "diabetes in pregnancy" and 3,126 men and women whose initial diagnosis resulted from the recording of a drug code unaccompanied by a medical code. This analysis led to no material difference in interpretation.
Comparison with other studies
Geiss et al. (3) found that overall relative mortality in type 2 diabetes was about twice that for subjects without diabetes. However, estimates from individual population-based studies ranged from 1.2 to 8.1 depending on age, sex, and country of origin. Many of these estimates derived from data collected from the 1980s or earlier. Mulnier et al. (2) also estimated a 2-fold higher mortality for diabetic subjects in GPRD compared with age- and sex-matched controls never diagnosed with diabetes. These were prevalent cases sampled from 1992 and may have been diagnosed some years earlier. Given a declining secular trend in relative mortality, the results of the present analyses are consistent with those of these earlier studies. Estimates of relative mortality in type 2 diabetes from other data sources will be of interest.
Conclusions
Relative mortality from type 2 diabetes is declining in men and women in the United Kingdom. As well as showing temporal trends, relative mortality from type 2 diabetes is also associated with age and duration of diabetes. It is possible that this decline is explained by improving long-term trends in incidence and mortality from cardiovascular disease, as well as by earlier diagnosis and improved case management, but the roles of the different explanations remain to be determined. Estimates of the overall future impact and burden of type 2 diabetes may need to be reconsidered. The extent to which improvements in relative mortality are equitably distributed also requires study.
| ACKNOWLEDGMENTS |
|---|
Author affiliation: Department of Public Health Sciences, Division of Health and Social Care Research, King's College London, London, United Kingdom (Martin C. Gulliford, Judith Charlton).
This study was funded by internal funds from the Department of Public Health Sciences. Access to the GPRD database was funded through the Medical Research Council's license agreement with MHRA.
The authors thank Dr. Paul Dickman for advice on statistical modeling.
Although this study is based in part on data from the Full Feature General Practice Research Database obtained under license from the United Kingdom MHRA, the interpretation and conclusions contained in this study are those of the authors alone.
Conflict of interest: none declared.
| References |
|---|
|
|
|---|
- Centers for Disease Control and Prevention. Crude and age-adjusted prevalence of diagnosed diabetes per 100 population, United States, 1980–2005. (2008) Atlanta, GA: Centers for Disease Control and Prevention. (http://www.cdc.gov/diabetes/statistics/prev/national/figage.htm). (Accessed February 21, 2008).
- Mulnier HE, Seaman HE, Raleigh VS, et al. Mortality in people with type 2 diabetes in the UK. Diabet Med (2006) 23(5):516–521.[CrossRef][Medline]
- Geiss LS, Herman WH, Smith PJ. Mortality in non-insulin-dependent diabetes mellitus. In: Diabetes in America—National Diabetes Data Group, ed. (2005) 2nd ed. Washington, DC: National Institutes of Health. 233–258.
- Franco OH, Steyerberg EW, Hu FB, et al. Associations of diabetes mellitus with total life expectancy and life expectancy with and without cardiovascular disease. Arch Intern Med (2007) 167(11):1145–1151.
[Abstract/Free Full Text] - Peeters A, Barendregt JJ, Willekens F, et al. Obesity in adulthood and its consequences for life expectancy: a life-table analysis. Ann Intern Med (2003) 138(1):24–32.
[Abstract/Free Full Text] - Boyle JP, Honeycutt AA, Narayan KM, et al. Projection of diabetes burden through 2050: impact of changing demography and disease prevalence in the U.S. Diabetes Care (2001) 24(11):1936–1940.
[Abstract/Free Full Text] - Ezzati M, Friedman AB, Kulkarni SC, et al. The reversal of fortunes: trends in county mortality and cross-county mortality disparities in the United States [electronic article]. PLoS Med (2008) 5(4):e66.[Medline]
- Thomas RJ, Palumbo PJ, Melton LJ III, et al. Trends in the mortality burden associated with diabetes mellitus: a population-based study in Rochester, Minn, 1970–1994. Arch Intern Med (2003) 163(4):445–451.
[Abstract/Free Full Text] - Gregg EW, Gu Q, Cheng YJ, et al. Mortality trends in men and women with diabetes, 1971 to 2000. Ann Intern Med (2007) 147(3):149–155.
[Abstract/Free Full Text] - Office for National Statistics. Mortality statistics, general. Review of the Register General on deaths in England and Wales, 2005. (Series DH1, no. 38). London, United Kingdom: Office for National Statistics; 2008. (http://www.statistics.gov.uk/downloads/theme_health/Dh1_38_2005/DH1_No_38.pdf). (Accessed April 30, 2008).
- National Center for Health Statistics. Death rates by age, sex, race, and underlying cause. United States, 1981–2004 (NMR04a). (2008) Washington, DC: National Center for Health Statistics. (http://205.207.175.93/aging/TableViewer/tableView.aspx?ReportId=424). (Accessed April 30, 2008).
- Lawson DH, Sherman V, Hollowell J. The General Practice Research Database. Scientific and Ethical Advisory Group. QJM (1998) 91(6):445–452.
[Abstract/Free Full Text] - Charlton J, Latinovic R, Gulliford MC. Explaining the decline in early mortality in men and women with type 2 diabetes: a population-based cohort study. Diabetes Care (2008) 31(9):1761–1766.
[Abstract/Free Full Text] - Shah AD, Martinez C. A comparison of the cause of death recorded in GPRD with national mortality statistics in England and Wales. Pharmacoepidemiol Drug Saf (2004) 13(suppl):S2–S3.
- Office for National Statistics. Interim Life Tables, United Kingdom, 1980–82 to 2004–06. London: United Kingdom. Office for National Statistics; 2008. (http://www.statistics.gov.uk/downloads/theme_population/Interim_Life/ILTUK0406Reg.xls). (Accessed February 21, 2008).
- Dickman PW, Coviello E, Hills M. Estimating and modelling relative survival. Stata J. In press.
- Dickman PW, Sloggett A, Hills M, et al. Regression models for relative survival. Stat Med (2004) 23(1):51–64.[CrossRef][Web of Science][Medline]
- Cooper R, Cutler J, Desvigne-Nickens P, et al. Trends and disparities in coronary heart disease, stroke, and other cardiovascular diseases in the United States: findings of the National Conference on Cardiovascular Disease Prevention. Circulation (2000) 102(25):3137–3147.
[Abstract/Free Full Text] - Ford ES, Capewell S. Coronary heart disease mortality among young adults in the U.S. from 1980 through 2002: concealed levelling of mortality rates. J Am Coll Cardiol (2007) 50(22):2128–2132.
[Abstract/Free Full Text] - O'Flaherty M, Ford E, Allender S, et al. Coronary heart disease trends in England and Wales from 1984 to 2004: concealed levelling of mortality rates among young adults. Heart (2008) 94(2):178–181.
[Abstract/Free Full Text] - Rautio A, Lundberg V, Messner T, et al. Favourable trends in the incidence and outcome of myocardial infarction in nondiabetic, but not in diabetic, subjects: findings from the MONICA myocardial infarction registry in northern Sweden in 1989–2000. J Intern Med (2005) 258(4):369–377.[CrossRef][Medline]
- American Diabetes Association. Standards of medical care in diabetes. Diabetes Care (2008) 31(suppl_1):S12–S54.
[Free Full Text] - Koopman RJ, Mainous AG III, Diaz VA, et al. Changes in age at diagnosis of type 2 diabetes mellitus in the United States, 1988 to 2000. Ann Fam Med (2005) 3(1):60–63.
[Abstract/Free Full Text] - Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). UK Prospective Diabetes Study Group. Lancet (1998) 352(9131):837–853.[CrossRef][Web of Science][Medline]
- Stevens RJ, Coleman RL, Adler AI, et al. Risk factors for myocardial infarction case fatality and stroke case fatality in type 2 diabetes: UKPDS 66. Diabetes Care (2004) 27(1):201–207.
[Abstract/Free Full Text] - Efficacy of atenolol and captopril in reducing risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 39. UK Prospective Diabetes Study Group. BMJ (1998) 317(7160):713–720.
[Abstract/Free Full Text] - MRC/BHF Heart Protection Study. Cholesterol-lowering with simvastatin in 5963 people with diabetes: a randomised placebo-controlled trial. Lancet (2003) 361(9326):2005–2016.[CrossRef][Web of Science][Medline]
- Colhoun HM, Betteridge DJ, Durrington PN, et al. Primary prevention of cardiovascular disease with atorvastatin in type 2 diabetes in the Collaborative Atorvastatin Diabetes Study (CARDS): multicentre randomised placebo-controlled trial. Lancet (2004) 364(9435):685–696.[CrossRef][Web of Science][Medline]
- Gaede P, Lund-Andersen H, Parving HH, et al. Effect of a multifactorial intervention on mortality in type 2 diabetes. N Engl J Med (2008) 358(6):580–591.
[Abstract/Free Full Text] - Gulliford MC, Latinovic R, Charlton J. Diabetes diagnosis, resource utilization, and health outcomes. Am J Manag Care (2008) 14(1):32–38.
- Gulliford MC, Charlton J, Latinovic R. Increased utilization of primary care 5 years before diagnosis of type 2 diabetes: a matched cohort study. Diabetes Care (2005) 28(1):47–52.
[Abstract/Free Full Text] - Greenland S, Rothman KJ. Introduction to stratified analysis. In: Modern Epidemiology—Rothman KJ, Greenland S, eds. (1998) 2nd ed. Philadelphia, PA: Lippincott Williams and Wilkins. 253–279.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
