Abstract
BACKGROUND: We aimed to test the performance of PRESERVE and RESP scores to predict death in patients with severe ARDS receiving extracorporeal membrane oxygenation (ECMO) with different case mixes.
METHODS: All consecutive patients treated with ECMO for refractory ARDS, regardless of cause, in the Caen University Hospital in northwestern France over the last decade were included in a retrospective cohort study. The receiver operating characteristic curves of each score were plotted, and the area under the curve was computed to assess their performance in predicting mortality (c-index).
RESULTS: Forty-one subjects were included. Pre-ECMO ventilator settings were: mean VT, 6.1 ± 0.9 mL/kg; breathing frequency, 32 ± 4 breaths/min; PEEP, 11 ± 4 cm H2O; peak inspiratory pressure, 48 ± 9 cm H2O; plateau pressure, 30.4 ± 4.4 cm H2O. At ECMO initiation, blood gas results were: pH 7.22 ± 0.17, PaO2/FIO2 = 63 ± 22 mm Hg; PaCO2 = 56 ± 18 mm Hg; FIO2 = 99 ± 2%. Pre-ECMO data were available in 35 and 27 subjects for calculation of the PRESERVE score and RESP score, respectively. Pre-ECMO scoring system results were: median PRESERVE score, 4 (interquartile range 2–5), and median RESP score, 0 (interquartile range −2 to 2). Twenty-three subjects (56%) died, including 19 receiving ECMO. In univariate analysis, plateau pressure (P = .031), driving pressure (P = <.001), and compliance (P = .02) recorded at the time of ECMO initiation as well as the PRESERVE score (P = .032) were significantly associated with mortality. With a c-index of 0.69 (95% CI 0.53–0.87), the PRESERVE score had better discrimination than the RESP score (c-index of 0.60 [95% CI 0.41–0.78]) for predicting mortality.
CONCLUSIONS: The use of these scores in helping physicians to determine the patients with ARDS most likely to benefit from ECMO should be limited in clinical practice because of their relatively poor performance in predicting death in subjects with severe ARDS receiving ECMO support. Before widespread use is initiated, these scoring systems should be tested in large prospective studies of subjects with severe ARDS undergoing ECMO treatment.
Introduction
Based on promising results from observational cohort studies during the influenza A (H1N1) pandemic1–3 and a positive randomized controlled trial,4 many centers have initiated extracorporeal membrane oxygenation (ECMO) programs as a salvage therapy for subjects with the most severe cases of ARDS who have refractory hypoxemia.5 However, the benefit of ECMO for survival remains controversial in this setting.6 Therefore, the selection of patients most likely to benefit from ECMO presents a difficult challenge for physicians. In this context, 2 mortality prediction scores (the PRESERVE [predicting death for severe ARDS on venovenous ECMO] score7 and the RESP [respiratory ECMO survival prediction] score8) have been developed based on large retrospective observational cohorts to help ICU physicians to select appropriate candidates for ECMO. However, external validation studies of these scores are scarce.9–13 The aim of this study was to test the performance of PRESERVE and RESP scores in predicting death in patients with severe ARDS, candidates for ECMO, with different case mixes.
QUICK LOOK
Current knowledge
The selection of patients with severe ARDS most likely to benefit from ECMO remains a difficult challenge for physicians. Based on large retrospective observational cohorts, 2 predictive mortality scores have been developed, the PRESERVE score and the RESP score, both having good performance in helping physicians to select the most appropriate candidates for ECMO.
What this paper contributes to our knowledge
In comparison with original cohorts, we report poor performance of these scores to predict death in subjects with ARDS receiving ECMO with different case mixes. Therefore, this study suggests that great caution is warranted when considering these scoring systems to individually select patients with ARDS for ECMO.
Methods
Subjects
We conducted a retrospective cohort study of all consecutive subjects treated with ECMO for refractory ARDS according to the ARDS Berlin definition 2011,14 regardless of cause, at the Caen University Hospital between January 2005 and September 2014. According to French legislation at the time of the study, and given that the study was observational and retrospective, no ethics committee approval was requested, and thus no informed consent was obtained from the subjects.
In our institution, ECMO was considered as a potential rescue treatment for subjects with severe ARDS and refractory hypoxemia unresponsive to a protective ventilation strategy as recommended in this setting.14 The decision to initiate ECMO is made collegially and in a multidisciplinary manner by urgent care specialists and cardiovascular surgeons in all cases.
Cannulation Technique and Management During ECMO
ECMO was performed only after admission into the Caen University Hospital, where medical teams and nurses have extensive experience with emergency extracorporeal life support.15–17 The hardware consisted of a Biomedicus (Medtronic, Minneapolis, Minnesota) or Maquet (Rotaflow, Jostra-Maquet, France) portable system incorporating a centrifugal pump console. The closed ECMO circuit consisted of preconnected polyvinyl chloride tubing (Medtronic or PLS 20-50 [Maquet]) having a hollow-fiber membrane oxygenator with an integrated heat exchanger. All components were heparin-coated.
All procedures were performed by trained cardiovascular surgeons. Venovenous ECMO was initiated with an Avalon monocannula (IST Cardiology, Caen, France), from 21 to 27 French, depending on the size of the subject, inserted into the right internal jugular vein using a modified Seldinger technique. The cannula was positioned to the distal suction port at the end of the inferior vena cava, the proximal suction port in the superior vena cava, and the injection port on the axis of the tricuspid valve in the right atrium, under echocardiographic guidance. Chest radiography was performed daily to monitor the cannula position. Some subjects were also treated using surgical cannulation in a femoro-jugular or femoro-femoral configuration (Biomedicus from 17 to 21 French). The pump speed was adjusted to obtain a blood flow of 2.5 L/m2. Intravenous unfractionated heparin was given to maintain the activated clotting time between 150 and 200 s.
The decision to discontinue ECMO was based upon evidence of irreversible multiple-organ failure. Subjects were weaned off of ECMO after lung recovery if the pulmonary function during a period of reduced pump flow and gas flow remained stable.
Retrospective Chart Review
Data collection focused specifically on demographics, comorbidities according to the Charlson index,18 severity of illness according to the Simplified Acute Physiology Score II19 and the Sequential Organ Failure Assessment20 at ICU admission, etiology of ARDS (ie, infectious pneumonia, trauma, peri-/postoperative, and other), and time to initiation of ECMO. At the time of ECMO initiation, records were kept of hemodynamic, neurological, renal (including extrarenal support), liver, hematological, and respiratory data. Respiratory data included: blood gas measurements, PaO2/FIO2, pre-/post-ECMO ventilator settings (ie, breathing frequency, tidal volume [VT], peak inspiratory pressure, PEEP, plateau pressure, driving pressure, and static compliance), and the use of adjunctive respiratory therapy (ie, inhaled nitric oxide, almitrine, neuromuscular blocking agents, and prone position). The Sequential Organ Failure Assessment score was also calculated on the day of ECMO initiation.
In addition, the RESP and PRESERVE scores were calculated according to the original papers.7,8 In the original cohorts based on multivariate logistic regression analysis, the PRESERVE score (0–14 points; higher score indicating higher probability of death) was constructed with 8 pre-ECMO subject characteristics (ie, age, body mass index, immunocompromised status, days of mechanical ventilation, sepsis-related organ failure assessment, plateau pressure, and PEEP), and the RESP score (−22 to 15 points; lower score indicating higher probability of death) with was constructed with 12 patient characteristics (ie, age, immunocompromised status, days of mechanical ventilation, diagnosis, central nervous system dysfunction, acute associated nonpulmonary infection, neuromuscular blocking agents or nitric oxide use, bicarbonate infusion, cardiac arrest, PaCO2, and peak inspiratory pressure). No assumptions were made for missing data. Therefore, scoring was performed only for subjects with all data recorded.
Each subject's clinical course during hospitalization was recorded, including ECMO duration and vascular, neurologic, hemorrhagic, renal, and perfusion system complications. Hospital mortality as well as survival at 6 months and 1 y and cause of death were recorded.
Statistical Analysis
Data were expressed as mean ± SD or median (interquartile range [IQR]) and percentage, depending on the type of variable of interest. Baselines were compared between groups (survivors and non-survivors) using the Fisher exact test with qualitative variables and using the unpaired Student t test or the Mann-Whitney test with continuous variables.
We performed univariate analysis at the time of ECMO implantation to identify baseline subject characteristics associated with hospital mortality and 6-month mortality. The small sample size of the study population precluded the performance of a valid multivariate analysis.
The receiver operating characteristic curves of the PRESERVE score and the RESP score were plotted, and the area under the curve was computed to assess their performance in predicting mortality (c-index). All tests were 2-sided, and a P value < .05 was considered statistically significant. The Hosmer and Lemeshow test was used to assess goodness of fit for both models.
Results
Subjects
During the study period, 41 subjects (12 women, 29 men, mean age 44 ± 18 y), including 7 subjects with an immunocompromised status, had been treated with ECMO rescue. The mean body mass index was 26.7 ± 7.2 kg/m2. The mean Charlson comorbidity index was 1 ± 2. The median (IQR) severity score at ICU admission was 50 (37–63) for the Simplified Acute Physiology Score II and 10 (6–14) for the Sequential Organ Failure Assessment score. The main reasons for ARDS were: infectious pneumonia (n = 20), trauma (n = 7), peri-/postoperative (n = 4), and others (n = 10).
Baseline Characteristics at ECMO Initiation
Baseline characteristics at ECMO initiation are shown in Table 1. ECMO was initiated 5 ± 4 d after mechanical ventilation via an endotracheal tube in all subjects. The median (IQR) Sequential Organ Failure Assessment score was 10 (8–13). One subject experienced out-of-hospital cardiac arrest, and 4 subjects experienced in-hospital cardiac arrest before ECMO implantation. Pre-ECMO ventilator settings were: mean ± SD VT, 6.1 ± 0.9 mL/kg; breathing frequency, 32 ± 4 breaths/min; PEEP, 11 ± 4 cm H2O; peak inspiratory pressure, 48 ± 9 cm H2O; plateau pressure, 30.4 ± 4.4 cm H2O. Thirty-five subjects (85%) received a vasopressor, and 27 subjects (64%) needed extrarenal support. All subjects were sedated and experienced refractory hypoxemia despite the recourse of neuromuscular blocking agents in all subjects. Prone positioning was performed in 15 subjects (37%), and inhaled nitric oxide and almitrine were administered in 35 subjects (85%) and 13 subjects (32%), respectively.
Baseline Characteristics at Extracorporeal Membrane Oxygenation Initiation
At ECMO initiation, blood test results were: pH 7.22 ± 0.17, PaO2/FIO2 = 63 ± 22 mm Hg (ie, 38 subjects with severe ARDS [PaO2/FIO2 < 100 mm Hg] and 3 subjects with mild ARDS [PaO2/FIO2 < 200 mm Hg]), PaCO2 = 56 ± 18 mm Hg, FIO2 = 99 ± 2%, serum bicarbonate = 23 ± 5 mmol/L, and plasma lactate = 3 ± 2 mmol/L.
All pre-ECMO data were available in 35 subjects and 27 subjects for calculation of the PRESERVE score and the RESP score, respectively. In these subjects, pre-ECMO scoring results were: median (IQR) PRESERVE score, 4 (2–5) and median RESP score, 0 (−2 to 2).
ECMO Characteristics
Venovenous ECMO was initiated with a monocannula in 28 subjects (68%) and with a double venovenous cannulation in 9 subjects (22%). In 4 subjects (10%), a veno-arterial cannulation was subsequently needed for associated refractory shock. The mean ± SD ECMO flow was 4.4 ± 1 L/min, and the sweep gas flow was 7 ± 3 L/min.
ECMO support allowed ultraprotective mechanical ventilator settings with a decrease in mean ± SD VT to 3.4 ± 1.4 mL/kg and an increase in mean PEEP to 14 ± 5 cm H2O, with a plateau pressure maintained < 30 cm H2O in all subjects. The mean ± SD length of ECMO therapy was 8 ± 6 d, and median (IQR) was 7 d (7–11 d).
Fourteen subjects (34%) experienced 14 severe ECMO-related complications (ie, 7 complications related to a double-lumen monocannula and 6 complications related to a double venovenous cannulation). The main significant ECMO-related complications were: 2 cardiac arrests (one ventricular fibrillation and one perforation of the right atrium) during the procedure and 10 cases of severe bleeding (one subarachnoid hemorrhage, one intracerebral hematoma, one hemorrhagic shock, one tamponade, one dissection of jugular vein, 3 hemothorax, and 2 severe hemoptysis), one case of inferior vena cava thrombus, and one case of filter thrombus during ECMO support).
Mortality and Risk Factors
Twenty-two subjects (54%) were weaned off of ECMO support alive. Twenty-three subjects (56%) died during their hospital stay: 19 receiving ECMO and 4 after ECMO weaning. In our cohort, 7 deaths (30%) were associated with a severe ECMO-related complication. The main reasons for the deaths were: septic shock with multiple organ failure (n = 6), refractory cardiac arrest (n = 1), refractory hypoxemia (n = 3), anoxic coma (n = 3), digestive ischemia (n = 2), hemothorax (n = 2), tanponade (n = 1), subarachnoid hemorrhage (n = 1), intracerebral hematoma (n = 1), hemorrhagic shock (n = 1), refractory intracranial hypertension (n = 1), and severe hemoptysis (n = 1).
Among subjects weaned off of ECMO support, 10 subjects needed a tracheostomy for weaning off of ventilator support. Almost all subjects discharged from the hospital alive were still alive at 6 months and at 1 y (except for one subject who died at month 7).
In univariate analysis, only plateau pressure (P = .031), driving pressure (P = <.001), and static compliance (P = .02) recorded at the time of ECMO initiation as well as the PRESERVE score (P = .03) were significantly associated with mortality, as shown in Table 1. However, a lower arterial pH, a lower PEEP, and a higher severity score tended to be associated with mortality.
Demographic data, number of days receiving mechanical ventilation before ECMO, and ARDS etiology were not associated with death. No difference was found in the data on ECMO management between survivors and non-survivors.
The percentage of survival in our cohort and original cohorts7,8 and other cohorts9–13 according to RESP and PRESERVE scores at ECMO initiation are shown in Table 2. The highest score risk class is consistently associated with the lowest probability of survival. In our study, the PRESERVE score was significantly higher in the group of deceased subjects: mean ± SD, 4.8 ± 2.7 versus 3.1 ± 1.8; median (IQR), 4 (2–7) versus 3 (1–5); P = .032. In contrast, the RESP score did not seem to be associated with mortality (Table 1). The performance of the PRESERVE and RESP scores for predicting mortality is reported in Figure 1, showing the receiver operating characteristic curve for predicting death. With a c-index of 0.69 (95% CI 0.53–0.87), the PRESERVE score demonstrated a better performance than the RESP score (c-index of 0.60 [95% CI 0.41–0.78]) for predicting mortality in subjects with severe ARDS receiving ECMO support The calibration test with the Hosmer and Lemeshow goodness-of-fit test for PRESERVE (P = .08) and RESP score (P = .89) indicated no significant differences between observed and predicted probabilities. The performance of PRESERVE and RESP scores in predicting death in subjects with ARDS receiving ECMO with different case mixes, reported in the literature, is shown in Table 3.
Percentage of Survival in Our Cohort and All Other Cohorts, Including Original Cohorts, Compared With RESP Score and PRESERVE Score at Extracorporeal Membrane Oxygenation Initiation
Receiver operating characteristics curves for PRESERVE score (A) and RESP score (B) for predicting mortality.
Survival Prediction Models in Subjects With Severe ARDS Treated With Extracorporeal Membrane Oxygenation
Discussion
The selection of patients with severe ARDS most likely to benefit from ECMO remains a difficult challenge for physicians. In this context, based on our experience, we confirm that the PRESERVE score7 seems to be more appropriate than the RESP score8 in selecting candidates for ECMO. However, the clinical impact of these scores is limited because of their relatively poor performance in predicting death, as reported in our cohort.
In this study, we report a hospital mortality and a 6-month mortality rate of 56% each. This result is consistent with the wide range of mortality (36–67%) reported by other authors.3,4,7,8,10,12,21–23 Differences in baseline characteristics of subjects at ECMO initiation (age, severity of illness, reasons for ARDS (influenza A [H1N1], bacterial pneumonia, others), time elapsed from mechanical ventilation to ECMO, pre-ECMO ventilator settings, and recourse to mobile ECMO could explain these differences. Mortality rate is clearly influenced by indications and contraindications defined in each center as shown in a previous study.24 Therefore, considering the paucity of randomized controlled studies in this field,4,21,25 the selection of ARDS patients most likely to benefit from ECMO remains a difficult challenge for physicians.
In this context, based on large observational cohorts, 2 predictive mortality scores have been constructed, the PRESERVE score7 and the RESP score,8 both having a good receiver operating characteristic curve performance in the original cohorts (c-index 0.89 [95% CI 083–0.94] for the PRESERVE score and a c-index of 0.74 [95% CI 0.72–0.76] for the RESP score) to help physicians select the most appropriate candidates for ECMO. In accordance with this study, we report that the performance of the PRESERVE score seems to be better than that of the RESP score in predicting death in subjects with ARDS receiving ECMO support.
Nevertheless, despite good external validation of the RESP score performed by the authors,8 the clinical relevance of these scores remains unclear. To our knowledge, only 5 other studies9–13 have tested the performance of these scores to predict death in subjects with severe ARDS receiving ECMO with different case mixes. Except for 2 studies,11,13 all studies9,10,12 reported a poor performance (c-index < 0.7) in accurately predicting mortality in this setting (Table 3). In comparison with original cohorts,7,8 we cannot exclude the possibility that differences in baseline subject characteristics in our cohort can partly explain these results. These characteristics could pertain to factors incorporated in the PRESERVE score calculation (lower prevalence of immunocompromised subjects and use of prone positioning before ECMO in our cohort) and the RESP score calculation (higher prevalence of pneumonia, immunocompromised subjects, use of inhaled nitric oxide and neuromuscular blocking agents, elevated peak inspiratory pressure, and longer time to ECMO). However, these differences should have only a limited impact on the performance of these scores, considering that the data were based on a large French (PRESERVE score) and international (RESP score) population with different case mixes. A unique aspect of our study is the high proportion (68%) of monocannula placement compared with the other studies (ranging from 0 to 25%).9–13 To our knowledge, no difference in performance between the types of cannulation is clearly reported in the literature.6 Therefore, the impact of the use of double-lumen monocannula on the performance of the models is probably low. In addition, based on the cumulative predicted survival for each risk class of these scores (Table 2), the proportion of subjects (ie, PRESERVE score ≥ 7 and RESP score ≤ −6) for whom the futility of recourse of ECMO could be discussed should be limited in clinical practice. Therefore, we believe that great caution is warranted when considering these scoring systems to individually select ARDS subjects for ECMO. Before widespread use is initiated, these scoring systems should be tested in large prospective studies of subjects with severe ARDS being treated with ECMO with different case mixes, as suggested by their authors.7,8
We are aware of study limitations. The single center retrospective observational design and the relatively small sample size of subjects used to perform the validation of each score calculated only in subjects with a complete data set may limit the interpretation and relevance of our data. Nevertheless, the recourse of ECMO in patients with ARDS in our clinical practice is in accordance with the most recent French recommendations in this field.6 In addition, our findings are in accordance with those of other studies,9–10,12 suggesting a good external validation of our work. Moreover, the high proportion of deaths associated with a severe ECMO-related complication, reported in our cohort, could limit the performance of a scoring system based on pre-ECMO patients' characteristics in predicting mortality. Unfortunately, this point was not addressed in original cohorts.7,8 Therefore, because of the absence of robust external validation of these scores, we believe that these data provide important information regarding the potential limits of these scoring systems in selecting patients most likely to benefit from ECMO.
Conclusions
This study suggests that the use of PRESERVE and RESP scores to help physicians determine the ARDS patients most likely to benefit from ECMO should be limited in clinical practice because of their relatively poor performance in predicting death in subjects with severe ARDS receiving ECMO support. Before widespread use is initiated, these scoring systems should be tested in large prospective studies of subjects with severe ARDS undergoing ECMO treatment.
Footnotes
- Correspondence: Cédric Daubin MD, Service de Réanimation Médicale, CHU de Caen, Av Côte de Nacre, 14033 Caen Cedex, France. E-mail: daubin-c{at}chu-caen.fr.
The authors have disclosed no conflicts of interest.
- Copyright © 2017 by Daedalus Enterprises
