Chest
Volume 141, Issue 1, January 2012, Pages 245-252
Journal home page for Chest

Postgraduate Education Corner
Contemporary Reviews In Critical Care Medicine
Severity Scoring in the Critically Ill: Part 1—Interpretation and Accuracy of Outcome Prediction Scoring Systems

https://doi.org/10.1378/chest.11-0330Get rights and content

This review examines the use of scoring systems to assess ICU performance. APACHE (Acute Physiology and Chronic Health Evaluation), MPM (mortality probability model), and SAPS (simplified acute physiology score) are the three major ICU scoring systems in use today. Central to all three is the use of physiologic data for severity adjustment. Differences in the size, nature, and time horizon of the data set translate into minor differences in accuracy and difficulty of data abstraction. APACHE IV provides ICU and hospital predictions for mortality and length of stay, whereas MPM and SAPS only provide hospital mortality predictions (although new algorithms generated from MPM data elements may predict ICU length of stay adequately). The primary use of scoring systems is for assessing ICU performance, with the ratio of actual-to-predicted outcomes in the study cohort providing performance comparisons to the reference ICUs. The reliability of scoring system predictions depends on the completeness and accuracy of the abstracted data; accordingly, ICUs must implement robust data quality control processes. CIs of the ratios are inversely related to sample size, and care must be taken to avoid overinterpreting changes in outcomes. ICU structural and process issues also can affect scoring system performance measures. Despite good discrimination and calibration, scoring systems are used in only 10% to 15% of US ICUs. Without ICU performance data, there is little hope of improving quality and reducing costs. Current demands for transparency and computerization of documentation are likely to drive future use of ICU scoring systems.

Section snippets

Comparison of ICU Scoring Systems

The three most commonly used ICU scoring systems are APACHE, MPM0 (where the 0 indicates from time of admission), and SAPS.3, 4, 5, 6, 7 Although the initial versions were introduced many years ago, each has undergone multiple revisions over the ensuing years.8, 9, 10 In addition to the regularly updated models introduced by the developers, other investigators have created alternate prognostic models using APACHE, SAPS, and MPM data elements that are customized to fit a specific patient

Evaluating Scoring Systems

Prior to discussing the performance of the different systems, several concepts bear brief review. First, it is important to understand how risk scoring systems are developed and evaluated. To generate a predictive model, most researchers divide their data set into two pools. The first is used to develop the model, generally through multivariable regression methods, whereas the second is reserved for validation after the final model is developed. Ideally, the performance in the development and

Data Rules and Data Quality

Data completeness rules and data quality can have major effects on outcome predictions and deserve major focus by all users of ICU scoring systems. Most ICU scoring systems generate predictions based on the data at hand, although APACHE will not generate predictions in the absence of certain data elements. Each system treats missing data as normal data. Afessa et al25 evaluated the impact of missing data on APACHE predictions and found worse outcomes in patients with missing data, suggesting

The Impact of ICU Structure and Care Processes on Score Interpretation

Variations in duration of care provided prior to admission to the ICU can introduce lead-time bias, which has been shown to affect severity-adjusted mortality rates.2, 27, 28, 29 Some EDs may define their primary mission as accurate triage and prompt transfer to the appropriate care locale, whereas others focus on ensuring optimal early treatment. The former approach results in patients possibly coming to the ICU with less treatment and, thus, more abnormal vital signs (eg, higher heart rates)

Barriers to Implementation

Currently, only 10% to 15% of US ICUs (by patient volume) use ICU scoring systems, which is clearly at odds with standard recommendations by quality experts in other fields for frequent measurement and close scrutiny of quality data.33 Commonly articulated obstacles to use include cost and concerns about accuracy and applicability to the patient population. Although annual licensing fees for APACHE dissuaded adoption by some hospitals in the past, open access to hospital mortality and ICU LOS

Conclusions

APACHE, MPM, and SAPS have evolved over the past 25 years in an effort to improve predictive accuracy and keep pace with evolving critical care practices. Each provides accurate hospital mortality predictions; APACHE (and potentially MPM) also generate LOS performance data. Given the dominant role of LOS in explaining ICU cost,35 hospital executives and ICU leaders will require data on this key financial and operational metric. Manual data collection burden is lower with MPM and SAPS. However,

Acknowledgments

Financial/nonfinancial disclosures: The authors have reported to CHEST the following conflicts of interest: Drs Breslow and Badawi are employees of Philips Healthcare Inc, where this work was performed.

Other contributions: We thank David Stone, MD, for his assistance in preparing this manuscript.

References (37)

  • RP Moreno et al.

    SAPS 3—from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission

    Intensive Care Med

    (2005)
  • TL Higgins et al.

    Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III)

    Crit Care Med

    (2007)
  • WA Knaus et al.

    APACHE-acute physiology and chronic health evaluation: a physiologically based classification system

    Crit Care Med

    (1981)
  • JR Le Gall et al.

    A simplified acute physiology score for ICU patients

    Crit Care Med

    (1984)
  • S Lemeshow et al.

    A method for predicting survival and mortality of ICU patients using objectively derived weights

    Crit Care Med

    (1985)
  • DA Harrison et al.

    A new risk prediction model for critical care: the Intensive Care National Audit & Research Centre (ICNARC) model

    Crit Care Med

    (2007)
  • DW de Lange et al.

    Severity of illness and outcome in ICU patients in the Netherlands: results from the NICE registry 2006–2007

    Neth J Crit Care

    (2009)
  • KM Rowan et al.

    Intensive Care Society's APACHE II study in Britain and Ireland—II: outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method

    BMJ

    (1993)
  • Cited by (157)

    • Ventilator-associated bacterial pneumonia in coronavirus 2019 disease, a retrospective monocentric cohort study

      2021, Journal of Infection and Chemotherapy
      Citation Excerpt :

      Furthermore, lung compliance was assessed in patients affected by bacterial VAP at diagnosis. Previously validated prognostic scores (Apache II, Apache III, SAPS 1 and SAPS 2) were calculated for all the included patients at 48 h from ICU admission [13]. Finally, sequential organ failure assessment (SOFA) score was documented for probable bacterial VAP patients at time of diagnosis, in order to estimate the prognosis and the severity of the disease [14].

    • Intensive care unit scoring systems

      2021, Critical Care Nurse
      Citation Excerpt :

      Nor can those values be compared with scores from other models developed on a different scale with different numbers of variables. Lead-time bias, defined as variations in care before ICU admission, can affect severity score values,42,43 with the variables of heart rate, blood pressure, respiratory rate, serum pH, oxygenation, and blood glucose level accounting for the most bias.13,44 For example, a patient with severe sepsis admitted from an emergency department capable of initiating optimal early treatment (eg, fluid resuscitation, antibiotic therapy, vasopressor support) will likely exhibit less vital sign derangement at ICU admission than a similar patient, emergently transferred from a general hospital unit, who has yet to be stabilized.

    • Diagnosing Medical Score Calculator Apps

      2023, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    View all citing articles on Scopus

    Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (http://www.chestpubs.org/site/misc/reprints.xhtml).

    View full text