Original Article
Accuracy of hospital morbidity data and the performance of comorbidity scores as predictors of mortality

https://doi.org/10.1016/j.jclinepi.2011.03.014Get rights and content

Abstract

Objectives

The main objectives of this study were to validate the hospital morbidity data (HMD) and to compare the performance of three comorbidity adjusting methods in predicting 1-year and 5-year all-cause mortality in a male general hospital population in Western Australia (WA).

Study Design and Setting

Population-based data were integrated with WA-linked data system. Deyo–Charlson Index, Enhanced-Charlson Index, and Elixhauser's method measured comorbidity. Mortality was modeled using Cox regression, and model discrimination was assessed by Harrell's C statistics.

Results

The HMD were most likely to identify major comorbidities, such as cancer, myocardial infarction, diabetes mellitus, and major operations. The presence of comorbidity was independently associated with an increased risk of adverse outcomes. All models achieved acceptable levels of discrimination (Harrell's C: 0.70–0.76). The Enhanced-Charlson Index matched the Deyo–Charlson Index in predicting mortality. Elixhauser's method outperformed the other two. Including information from past admissions achieved nonsignificant improvement in model discrimination. A dose-response effect was observed in the effect of repeated episodes on risk of 5-year mortality.

Conclusion

Comorbidities diagnosed at different points in time may have different associations with the risk of adverse outcomes. More research is required to integrate the effect of repeated episodes in currently used methods that measure and adjust for comorbidity.

Introduction

What is new and what should change now?

  • Presence of comorbidity, irrespective of method used to measure it, was independently associated with an increased risk of 1-year and 5-year mortality.

  • The Enhanced-Charlson Index matched the original Deyo–Charlson Index in predicting mortality.

  • Retrieving information on comorbidity from previous hospital admissions significantly increased the prevalence of patients' comorbidities but did not result in any improvement in model discrimination. Thus information from any index hospitalization could be sufficient to measure and adjust for comorbidity.

  • The Western Australia hospital morbidity data (HMD) system is a valid tool to assess major outcomes.

  • A dose-response effect of repeated conditions (e.g., myocardial infarction and cerebrovascular accident) as reported in the administrative HMD was demonstrated on the risk of 5-year mortality.

  • The findings suggest that comorbid conditions diagnosed at different points in time may have different associations with the risk of dying.

  • More research is required to integrate the effect of repeated episodes in currently used methods that measure and adjust for comorbidity.

Monitoring systems often use administrative data (e.g., hospital morbidity data [HMD]) to predict, at the time of hospital admission, each patient's probability of developing an adverse outcome if average care were given [1], [2]. Differences in outcome among patients may or may not indicate differences in the quality of care that the patients received because these differences may be attributed to many factors, including differences in age, severity of illness, comorbid conditions, and also differences in methods of data collection and data quality [3], [4], [5], [6], [7], [8].

The power of any model to predict an adverse outcome depends on the extent and accuracy of the data on each patient's clinical condition when care began [9]. Although clinical data retrieved from patients' files (or chart review) are considered the gold standard for accurate clinical information, these are costly and time consuming to obtain, and often large clinical databases for comparative purposes are not easily available. Therefore, administrative data or claims data are being increasingly used to assess clinical outcomes and monitor, evaluate, and improve the quality of care [10], [11]. These data have many advantages over the clinical data derived from chart review, including their availability and coverage of large populations. However, research based on administrative data, such as HMD, requires a rigorous assessment of their quality, so considerable effort must be invested in data validation [4], [12], [13], [14]. This is particularly true given that the HMD were not originally collected for the purpose of health research. Another major disadvantage of HMD is the difficulty of differentiating complications from coexisting conditions [15], particularly if a single hospital admission (or index admission) rather than all previous hospital admissions were used to derive the medical information. Extraction of comorbidity information that relies on a single hospital admission may underestimate a patient's comorbid status [16], [17]. This becomes evident when an attending physician who summarizes the discharge document chooses to document more serious clinical conditions, while leaving out “less serious” diagnoses, such as chronic conditions [9]. This is especially true in administrative data sets that have limited coding spaces [18]. Incomplete coding of secondary diagnoses may result in inadequate adjustment for illness severity because of differences in case-mix reporting, which may bias assessments of patient risks of poor outcomes, as demonstrated in a large study on in-hospital mortality among elderly patients [9]. Iezzoni et al. found that chronic conditions, such as adult-onset diabetes mellitus (DM), hypertension, angina pectoris, heart valve disease, and previous myocardial infarction (MI) significantly lowered the risk of in-hospital mortality. Because on a clinical basis, these conditions were expected to increase the risk of death, the researchers concluded that their findings were probably because of undercoding of chronic conditions for more severely ill patients [9].

In hospitalized populations, comorbidity is one of the main factors associated with adverse outcomes [5], [7], [19], [20], [21], [22]. Studies have consistently shown that comorbidity (either acute or chronic illness) predicts higher in-hospital mortality, longer length of hospital stay, higher hospital costs, and readmissions [19], [20], [21], [22]. Thus, many researchers have developed coding algorithms that suit administrative data to account for patients' comorbidity conditions [23], [24], [25], [26], [27], [28], [29], [30]. Among the International Classification of Diseases (ICD) coding algorithms, Charlson Comorbidity Index (CCI) [30] with its many adaptations [24], [25], [26], [27], [28] and Elixhauser's comorbidities [29] are the most widely used in administrative data sets to measure and control for the effects of comorbid illness. Initially developed in 1987 [30], the CCI predicts major outcomes (mainly 1-year all-cause mortality) for patients who may have up to 19 comorbid conditions. Each condition is assigned a score of 1, 2, 3, or 6 depending on the risk of dying associated with the condition. The scores are summed to give a total score that predicts the event of interest. In contrast, Elixhauser's 30 comorbidities do not sum to an index but are retained as separate and independent measures, thus allowing the assessment of the association of different comorbid conditions with different outcomes [29].

The objectives of this study that integrated longitudinal data from a large population-based cohort of men with Western Australia (WA) HMD, Cancer Registry, Mental Health Services System (MHSS), and mortality records were the following: (1) to validate the HMD of WA, (2) to compare the performance of three comorbidity adjusting methods in predicting 1-year and 5-year all-cause mortality in a male elderly general hospital population in WA, (3) to compare levels of model discrimination using information derived from a single index hospitalization with that derived from all past hospital admissions, (4) to assess the association of comorbidity with 1-year and 5-year mortality and with any hospitalization after baseline screening, and (5) to assess the associations of past repeated conditions with mortality.

Section snippets

Data sources and study population

The study population comes from the Health In Men Study (HIMS) cohort [31], [32], which was originally established via a randomized population-based trial of ultrasound screening for abdominal aortic aneurysms (AAA) in men aged 65–83 living in Perth, WA in 1996–99. A total of 41,000 men was identified via the electoral roll (voting is compulsory in Australia) and was randomized into invited and control groups of equal size. Of the 19,352 men who were invited, 12,203 attended the baseline

Study population

A total of 12,203 men (mean age ± SD 72.1 ± 4.4 years) participated in the baseline AAA screening study (Table 1), of whom 12,013 (98.4%) had at least one hospital admission during 1970–2007. Of these, 10,950 men (91.1%) had already been hospitalized at least once before baseline screening. The HMD for these 10,950 men contained 66,344 hospitalizations with a median of four hospitalizations per person.

Validation

The HMD were most likely to identify cancer for any site except for melanoma, when the Cancer

Discussion

This study linked longitudinal clinical data from a large population-based cohort with administrative data sets, including HMD, Cancer Registry, MHSS, and mortality records. For each participant, any significant morbidity or health-related outcome was retrieved from the linked data in the period 1979 through to 2007. The linkage with clinical data enabled us to validate lifetime comorbidities as listed in the WA HMD. Several studies have attempted to validate diagnoses reported in

Conclusion

Our study shows that the HMD can be a valid tool to assess major outcomes among the general hospitalized population. The presence of comorbidity as noted in these data significantly and independently predicts short- and long-term major outcomes. Comorbidities diagnosed at different points in time may have different associations with the risk of adverse outcomes. More research is required to integrate the effect of repeated episodes in currently used methods that measure and adjust for

Acknowledgment

The authors pay tribute to the late Professor Konrad Jamrozik who made a significant contribution to the initiation and design of this study.

References (45)

  • P. Aylin et al.

    Use of administrative data or clinical databases as predictors of risk of death in hospital: comparison of models

    BMJ

    (2007)
  • A.K. Rosen et al.

    Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data?

    Med Care

    (2005)
  • J.B. Dimick et al.

    Surgical mortality as an indicator of hospital quality: the problem with small sample size

    JAMA

    (2004)
  • R.A. Hayward et al.

    Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer

    JAMA

    (2001)
  • S. Greenfield et al.

    The importance of co-existent disease in the occurrence of postoperative complications and one-year recovery in patients undergoing total hip replacement. Comorbidity and outcomes after hip replacement

    Med Care

    (1993)
  • J.G. Jollis et al.

    Discordance of databases designed for claims payment versus clinical information systems. Implications for outcomes research

    Ann Intern Med

    (1993)
  • T.L. Higgins et al.

    Early indicators of prolonged intensive care unit stay: impact of illness severity, physician staffing, and pre-intensive care unit length of stay

    Crit Care Med

    (2003)
  • J.B. Dimick et al.

    Should older patients be selectively referred to high-volume centers for abdominal aortic surgery?

    Vascular

    (2004)
  • L.I. Iezzoni et al.

    Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality?

    JAMA

    (1992)
  • J.B. Mitchell et al.

    Using Medicare claims for outcomes research

    Med Care

    (1994)
  • C. Zhan et al.

    Administrative data based patient safety research: a critical review

    Qual Saf Health Care

    (2003)
  • E.S. Fisher et al.

    The accuracy of Medicare's hospital claims data: progress has been made, but problems remain

    Am J Public Health

    (1992)
  • Cited by (63)

    • Medication burden in epilepsy: Exploring the impact of non-epilepsy concomitant drugs load

      2020, Seizure
      Citation Excerpt :

      A strength of the Rx Risk index, however, is that it is based on objective drug dispensing information rather than clinical coding which carries the risk of underestimation. Comorbidities are often underreported in hospital databases, especially for diagnoses for which hospitalization is not always necessary, e.g. asthma, hypertension and some psychiatric disorders [32,33]. Coding also often requires a clinic visit while some comorbidities are primarily treated outside hospitals.

    • Comparison of intravenous and oral definitive antibiotic regimens in hospitalised patients with Gram-negative bacteraemia from a urinary tract infection

      2019, Journal of Global Antimicrobial Resistance
      Citation Excerpt :

      Clinical data (hospital admission/discharge dates, Charlson comorbidity index, maximum body temperature within 24 h if transitioned from IV to oral therapy), pharmacological data (definitive antibiotic selection, duration of therapy) and microbiology data [dates of positive and first negative blood culture (if available), organism identified, antimicrobial susceptibilities] were also collected. The severity and age-weighted Charlson comorbidity index was computed based on the date of first positive blood culture for all included patients to assess the severity of illness [7]. Study data were collected and managed using Microsoft Excel (Microsoft Corp., Redmond, WA) and REDCap (Research Electronic Data Capture), a secure, Web-based application designed to support data capture for research studies [8].

    View all citing articles on Scopus
    View full text