Abstract
BACKGROUND: Timing of intubation in COVID-19 is controversial. We sought to determine the association of the ROX (Respiratory rate–OXygenation) index defined as divided by divided by breathing frequency at the time of intubation with clinical outcomes.
METHODS: We conducted a retrospective cohort study of patients with COVID-19 who were intubated by using a database composed of electronic health record data from patients with COVID-19 from 62 institutions. Multivariable logistic regression was used to evaluate the impact of ROX index score on mortality. We analyzed the ROX index as a continuous variable as well as a categorical variable by using cutoffs previously described as predicting success with high-flow nasal cannula.
RESULTS: Of 1,087 subjects in the analysis group, the median age was 64 years, and more than half had diabetes; 55.2% died, 1.8% were discharged to hospice, 7.8% were discharged to home, 27.3% were discharged to another institution, and 7.8% had another disposition. Increasing age and a longer time from admission to intubation were associated with mortality. After adjusting for sex, race, age, comorbidities, and days from admission to intubation, an increasing ROX index score at the time of intubation was associated with a lower risk of death. In a logistic regression model, each increase in the ROX index score by 1 at the time of intubation was associated with an 8% reduction in odds of mortality (odds ratio 0.92, 95% CI 0.88–0.95). We also found an odds ratio for death of 0.62 (95% CI 0.47–0.81) for subjects with an ROX index score ≥ 4.88 at the time of intubation.
CONCLUSIONS: Among a cohort of subjects with COVID-19 who were ultimately intubated, a higher ROX index at the time of intubation was positively associated with survival.
Introduction
The timing of intubation is an important and controversial decision in patients with COVID-19 and respiratory failure. If intubated too early, then patients, who otherwise would not have required mechanical ventilation, are exposed to complications such as sedation, delirium, and ventilator-associated pneumonia.1,2 Clinicians often use noninvasive ventilation in an effort to avoid these complications. However, noninvasive ventilation is known to cause ventilator-induced lung injury through many of the same mechanisms, such as volutrauma and biotrauma, that are seen in patients invasively ventilated.3 In biotrauma, cytokines released from injured lung not only cause worsening of lung injury but can promote dysfunction in other organs, for example, the kidneys.4,5 Patients receiving noninvasive ventilation may experience elevated transpulmonary pressures, high tidal volumes, pendelluft, increased intravascular pressure, and asynchrony, which result in patient-induced lung injury.3,6,7 Patient-induced lung injury has also been postulated to occur in patients with severe lung injury breathing spontaneously without positive pressure.7
The optimal oxygenation target among patients in the ICU also remains controversial.8 Also, patients on prolonged noninvasive ventilation may not receive adequate nutritional support and may develop facial skin breakdown.9-11 Expert clinicians have advocated approaches ranging from extremely late to very early intubation.12-14 The popularity of the different recommendations varies from institution to institution and has evolved throughout the pandemic. In addition to respiratory management, other patient- and treatment-related factors influence outcomes. Cardiovascular disease, diabetes, obesity, and older age have been associated with a higher risk of death from COVID-19 in multiple studies.15-18 Acute kidney injury is both common in hospitalized patients with COVID-19 and associated with higher mortality.19 Pharmacologic treatments such as the antiviral remdesivir and anti-inflammatory treatments (eg, steroids) have been shown to improve outcomes in patients with COVID-19 and with respiratory failure.20,21 Evidence with regard to the timing of intubation in COVID-19 is mostly based on observational cohort studies, which, not surprisingly, reached different conclusions.22-26
A retrospective cohort study, at 5 New York City hospitals, of 755 subjects who were intubated, reported that increasing time from hospital admission to intubation was associated with a small but statistically significant increased risk of death.22 In contrast, a prospective cohort study from France of 245 subjects who were invasively ventilated found that intubation > 2 d after admission was associated with improved survival.23 A single-center study of 75 mechanically ventilated subjects reported that late intubation (>1.27 d) was associated with a longer duration of mechanical ventilation, lower lung compliance, and worse survival.24 A meta-analysis of 12 studies of 8,944 subjects with COVID-19 reported that there was no difference in survival with early versus late intubation.25 An analysis of 231 subjects at 4 United States hospitals found that mortality was ∼38% among subjects intubated within 8 h of admission and was the same for those intubated > 24 h after admission.26 Investigators in Greece are currently recruiting participants for a randomized trial to address the question of early versus late intubation in COVID-19 (NCT04632043).
Early and late intubations exist on a continuum of respiratory distress. The degree of respiratory distress can be approximated by the ROX (Respiratory rate – OXygenation) index ([/]/breathing frequency). The ROX index was first validated by Rocca et al27 in a cohort of 191 subjects with bacterial pneumonia and on high-flow nasal cannula (HFNC). They examined ROX scores at 2, 6, and 12 h after initiation of HFNC; and reported that scores ≥ 4.88 were associated with a lower risk of intubation. Early in the pandemic, this same group examined the use of the ROX index for subjects with COVID-19 and on HFNC and found that a slightly higher cutoff, of ≥ 5.37, within 4 h of HFNC initiation was associated with a lower risk of intubation.28 Subsequently, various investigators have studied the use of the ROX index in COVID-19 in different cohorts at varying times in the disease course, resulting in slightly different proposed cutoffs. In a cohort of 113 subjects treated with or without HFNC, Colaianni-Alfonso et al29 report a ROX index score cutoff of ≤ 6.28 after 12 h of treatment predicted failure of noninvasive therapy with sensitivity of 97.6% and specificity of 51.8%. In a cohort of 120 subjects, Vega et al30 reported that, after 12 h of HFNC therapy, the optimal ROX cutoff was 5.99. Myers et al31 suggested a ROX cutoff at 12 h of 3.85 for predicting the need for mechanical ventilation in COVID-19. Chandel et al32 focused on using the ROX index to predict success of HFNC, rather than failure, in subjects with COVID-19, and found that, among those not intubated or weaned from HFNC after 12 h, a ROX index score > 3.0 had a sensitivity of 85.3% and specificity of 51.1% for successful treatment (ie, not needing intubation).
In studies of subjects with COVID-19, the range of suggested ROX cutoff values varies from 1.4 in an emergency department cohort to 25.6 in hospitalized patients.33,34 A meta-analysis of studies that examined the use of the ROX index for subjects with COVID-19 and on HFNC concluded that “the optimal cutoff value may fall close to 5.”35 These data demonstrate the utility of the ROX index in COVID-19 respiratory failure; however, for the clinician at the bedside, applying this finding still involves a substantial amount of art given the variability in populations studied, timing in the illness course, suggested ROX index cutoff values, and the relatively low specificity. Although the ROX index was initially reported as a method for determining if patients with bacterial pneumonia treated with HFNC were likely to need intubation, its components make it an intuitive measure of severity of respiratory distress in general. The components of the ROX index are degrees of hypoxemia and tachypnea, which have historically been used subjectively by clinicians to make decisions with regard to the need for intubation.
We hypothesized that, if a high ROX index value (low respiratory distress) at the time of intubation is associated with better outcomes, then this would favor early intubation. If not, it would suggest that early intubation may needlessly subject patients to complications of invasive mechanical ventilation. Therefore, we sought to determine the association of the ROX index score at the time of intubation with clinical outcomes by using Cerner Real-World Data (Cerner Corporation, North Kansas City, MO), a large database built from electronic health records organized into a common data model and managed by Cerner. Given that acute kidney injury is known to increase the risk of mortality, we also sought to determine the impact of acute kidney injury on the relationship between the ROX index and mortality.
Quick Look
Current Knowledge
The timing of intubation in patients with COVID-19 is controversial. Although some clinicians use similar criteria and oxygenation cutoffs as in other causes of respiratory failure, others have argued that COVID-19 should be managed differently.
What This Paper Contributes to Our Knowledge
We found that, among subjects who ultimately underwent endotracheal intubation, those intubated at lower respiratory severity illness as measured by ROX index scores had better outcomes. This remained true in sensitivity analyses that accounted for non-respiratory severity of illness.
Methods
This retrospective cohort study was approved by the Christiana Care institutional review board (CCC 40097) on June 17, 2020.
Study Population
We created a dataset by using the Cerner COVID Data Lab database released on September 2, 2020. This database contains deidentified electronic health record data on patients with COVID-19 from 62 institutions and has previously been described in detail.36 Criteria for inclusion were an in-patient hospital encounter that had a diagnosis of COVID during which the patient was intubated. We defined the diagnosis of COVID-19 as a positive COVID-19 test result during the encounter or within 2 weeks before the encounter. We excluded patients intubated in the emergency department. We excluded patients with missing data that did not allow for the calculation of the ROX index score or who did not have a ROX index score within 4 h of intubation. Encounters before the diagnosis of COVID-19 were also included from January 1, 2015, onward for all the subjects who met the inclusion criteria. Most of the subjects in this dataset were hospitalized in the first 6 months of 2020. However, because the dates were shifted to protect confidentiality, we cannot verify the exact hospitalization dates. A consort diagram that shows the creation of the analysis dataset is presented in Figure 1.
Intubation and Intubation Time
The following International Classification of Diseases, Tenth Revision, Procedure Coding System procedure codes were used to identify the subjects who underwent intuba-tion and mechanical ventilation: 5A1945Z, 5A1955Z, and OBH17EZ (obtained via http://cms.gov, Accessed May 22, 2022). The time of intubation was designated as the earliest of the service date/time for the above procedure or earliest date/time for ventilator settings, whichever was earliest. The codes used to identify ventilator settings in the Cerner database are shown in Appendix 1 (see the supplementary materials at http://www.rcjournal.com).
Oxygen Source and the ROX Index Calculation
From the database, we obtained the oxygen source (ie, nasal cannula, Non-rebreathing mask), breathing frequency, , and oxygen flows as well as when available. When was available, the ROX index score was calculated as reported divided by reported divided by breathing frequency. When an was not reported but nasal cannula oxygen flows were between 1 and 6 L/min, was calculated according to the estimates provided by Fuentes et al37 (Appendix 2, see the supplementary materials at http://www.rcjournal.com) and was used to calculate the ROX index score. Examples of ROX calculation are shown in Appendix 3 (see the supplementary materials at http://www.rcjournal.com). We defined high-flow oxygen as ≥ 30 L/min. For our analysis, we used the ROX index score closest to but preceding the time of intubation.
Kidney Function
We used serum creatinine values to evaluate acute changes in kidney function during hospitalization. We defined acute kidney injury as an increase in serum creatinine > 0.3 mg/dL.38 We then examined the relationship between the ROX index score and mortality for patients with and without acute kidney injury.
Severity of Illness
We calculated a modified sequential organ failure assessment (SOFA) score by excluding the respiratory system component because the respiratory component is part of the ROX index calculation. We calculated the modified SOFA score by using the worst values within 24 h of intubation of all non-respiratory components (coagulation, liver, cardiovascular, central nervous system, and renal). We used previously described numerical scores for each of the components.39,40 Although modifying the SOFA score makes the total score not comparable with other studies that reported an unmodified SOFA score, our goal in using the modified SOFA score was to control for non-respiratory severity illness, not to predict mortality overall. This method of removing the respiratory component of the SOFA score has been previously used by investigators seeking to look separately at respiratory and the non-respiratory severity of illness.41
Statistical Analysis
Bivariate comparisons of subjects included versus patients excluded from analyses were conducted by using the Wilcoxon rank-sum or chi-square tests. The estimated probability of in-hospital death from logistic regression was plotted against the polynomial ROX index score to investigate the relationship between the ROX index score and mortality, and the effect of acute kidney injury on this relationship after examining the interaction between the ROX index score and acute kidney injury. Subjects’ probability of mortality was graphed, along with the 95% CI. A multivariable logistic regression model evaluated the impact of the ROX index on in-hospital mortality, adjusting for sex, race, age, comorbidities, and days from admission to intubation. Additional multivariable logistic regression models were constructed to examine the effect of the ROX index on mortality after adjusting for a modified SOFA score and for the Glasgow coma scale score. All models that used mortality used in-patient mortality alone and did not include subjects who may have died after discharge. Statistical analyses were conducted by using R Version 4.2 (R Foundation for Statistical Computing, Vienna, Austria).
Results
The dataset contained 2,102 patients. Due to missing data, the ROX index score before intubation could only be calculated on 1,690 patients. An additional 603 patients were excluded because the last ROX index score available before intubation was > 4 h before intubation, which resulted in an analysis cohort of 1,087 subjects. The median (interquartile range [IQR]) time between the calculated ROX index score and the time of intubation was 0.95 (0.5–1.0) h and the median (IQR) number of days from admission to intubation was 1.37 (0.15–3.50) d. Characteristics of those included and excluded from the analysis were similar, as shown in Table 1. Diabetes and hypertension were extremely common comorbidities in this population. The term “high-flow nasal cannula” does not appear in the dataset, and the subjects on nasal cannula predominately had flows < 6 L/min. Only 33 subjects had flow ≥ 30 L/min. The last recorded oxygen source before intubation is shown in Appendix 4 (see the supplementary materials at http://www.rcjournal.com).
The discharge dispositions of subjects included and those excluded are shown in Table 2. In our final study population of 1,087 subjects who were intubated, 85 subjects (7.8%) were discharged home, 287 (27.3%) were discharged to a facility such as a rehabilitation facility or a nursing home, 600 (55.2%) died, 20 (1.8%) were discharged to hospice, and 85 (7.8%) had another discharge disposition. Creatinine values were available for 976 subjects (89.8%) . The median (IQR) creatinine level at the time of intubation was 1.40 (0.9–2.5) mg/dL. Most subjects (74%) were on at least 1 vasopressor at some point during their hospital stay. The median (IQR) ROX index score before intubation was 4.0 (3.0–5.9). The median (IQR) number of ventilator days was 9 (5–17) d. There were 15 subjects (1.4%) treated with extracorporeal membrane oxygenation.
After adjusting for sex, race, age, comorbidities, days from admission to intubation, an increasing ROX index score at the time of intubation was associated with a lower risk of death (Table 3). We found that each increase in the ROX index score of 1.0 before intubation was associated with an 8% reduction in odds of death in the hospital (OR 0.92, 95% CI 0.88–0.95; P < .001). The linear relationship between the ROX index score before intubation and estimated probability of death is shown in Figure 2. We also examined the odds of death for ROX index values > 4.88 and 5.36, which have been reported to predict successful management with HFNC (ie, without need for intubation) in subjects with pneumonia due to bacteria and COVID-19 respectively.27,28,42 We found a 38% reduction in the odds of death for the subjects with a ROX index score ≥ 4.88 (OR 0.62, 95% CI 0.47–0.81; P < .001) and a 42% reduction in the odds of death for the subjects with a ROX index score of ≥5.36 (OR 0.58, 95% CI 0.44–0.77; P < .001). Increasing age and each day from admission to intubation were associated with an increase in mortality (Table 3).
Of 976 subjects with creatinine values, 409 (41.9%) had a maximum increase in creatinine level at any time after intubation of ≤ 0.3 mg/dL, 181 (18.5%) had an increase of >0.3 and ≤ 1.0 mg/dL, 256 (26.2%) had an increase >1 to ≤4 mg/dL, and 130 (13.3%) had an increase of >4 mg/dL. The ROX index score at the time of intubation was not associated with an increase in creatinine but acute kidney injury (defined as an increase in creatinine level of ≥ 0.3 mg/dL) developing after intubation was associated with increased mortality (OR 4.34, 95% CI 3.24–5.84; P < .001). The association between the ROX index score and mortality was similar for subjects with and without acute kidney injury, as shown in Figure 3. The subgroup of subjects with an increase in creatinine of > 4 mg/dL (n = 130) had a survival of 27.5% compared with a 64.1% survival for subjects with an increase in creatinine ≤0.3 mg/dL (n = 409).
We conducted a sensitivity analysis that excluded subjects with missing values for creatinine, the Glasgow coma scale score, platelets, or bilirubin, which resulted in 569 subjects available for analysis. Because of this important decrease in sample size, we opted not to include the modified SOFA score in the multivariable logistic regression model reported above. However, we did analyze this subset for association between the ROX index score and mortality after controlling for sex, age, race, modified SOFA score, and days to intubation as covariates. As shown in Table 4, the OR for mortality with an increasing ROX index score was similar at 0.95 (95% CI 0.90–1.01; P = .09). The modified SOFA score was associated with mortality (OR 1.14, 95% CI 1.06–1.22 for each point increase). We also created a model that included 943 patients with the Glasgow coma scale score available to examine the impact of neurologic status on the association of the ROX index score and mortality. As shown in Table 5, the OR for the ROX index score in this model was 0.92, 95% CI 0.87–0.96.
Discussion
A higher ROX index score at the time of intubation was associated with improved survival to hospital discharge. This may reflect a higher severity of respiratory acuity in the subjects with a lower ROX index score. However, this finding remained consistent, even when adjusting for multiple variables and comorbidities, which suggests that delaying intubation until a higher respiratory severity of illness as measured by a decreased ROX index score may be associated with higher mortality. The association between increasing days from admission to intubation and increasing mortality also raises concerns about the risks of delaying intubation. It also provides some reassurance that earlier intubation is not associated with harm. Although the ROX index score at the time of intubation was not associated with the development of acute kidney injury, we did confirm previous work19 that suggests that acute kidney injury in COVID-19 results in substantially worse prognosis, which suggests that every effort should be made to avoid this complication. The developers of the ROX index validated subjects with respiratory failure caused by pneumonia and treated with HFNC.41 Therefore, the cutoff values that predict success or failure of conservative management without intubation are not necessarily valid in the subjects with predominantly lower-flow supplemental oxygen in our study. Nonetheless, it is worth noting that, when the subjects were intubated at or above ROX index scores previously reported to predict success with HFNC, outcomes were better.
The ROX index score captures numerically what many clinicians use in considering when to intubate a patient (ie, oxygenation and work of breathing). Therefore, a high ROX index score suggests a low threshold (decision for early intubation), and a lower ROX index score suggests high threshold (decision for late intubation). In both the lay press and medical literature, there has been discussion of so-called happy hypoxemia in COVID-19.43-45 Some critical care physicians have hypothesized that ARDS from COVID-19 is different and should be treated differently than ARDS from other respiratory viruses, for example, influenza. In our experience, the absence of symptoms combined with the high volume of patients with COVID-19 has caused some clinicians to use 100% oxygen delivered by noninvasive devices for longer and tolerate lower than they would in other diseases, for example, influenza.
Our multi-center retrospective study confirmed the results from a previous single-center study that suggested worse clinical outcomes with lower ROX index scores.46 Patients with a higher ROX index score may do better because they have a lower respiratory specific severity illness and would not have progressed to a worse ROX index score had they not been intubated. However, a recent prospective cohort study of 84 subjects with ARDS due to COVID-19 in the ICU found that respiratory distress, defined as breathing frequency > 25 breaths/min and / < 100 mm Hg before intubation, was strongly associated with an inability to achieve driving pressure < 14 cm H2O during the first 24 h of lung-protective ventilation.47 This study suggests that, at least in subjects who eventually require intubation, delaying intubation beyond thresholds typically used for other causes of respiratory failure may result in worse respiratory mechanics as assessed by driving pressure. Driving pressure is known to be associated with mortality.48 Our study adds to these findings by demonstrating that delayed intubation may be associated with worse clinical outcomes. Further prospective randomized controlled trials are needed to determine the optimal timing of intubation. Strengths of this analysis include the large dataset of Cerner-based electronic health records. Although treatments may vary across institutions, the multi-center design should eliminate bias from an individual health system. In addition, our findings remained consistent after including a marker of severity of disease (modified SOFA) in a sensitivity analysis.
Several limitations can be found in this retrospective cohort study. The Cerner COVID Data Lab database is composed of data collected for patient care, not research. Therefore, the data dictionary that explains what each of the entries means is limited. Different institutions may describe the same intervention differently in their electronic health records and document events or interventions based on varied local policies. Of note, few subjects received HFNC before intubation. This could be due to misclassification of oxygen delivery devices (eg, a ventilator used for high flow) or missing data, but it may reflect the timing of when the subjects were treated during the pandemic because, early in the pandemic, there were fears that HFNC might increase infection transmission. The dataset we used was released in September of 2020, and, although we are not able to determine when during the pandemic the subjects were hospitalized, most were probably treated during the first 6 months of 2020.
In addition, due to date shifting to protect confidentiality, we could not examine trends over time. We used an accepted conversion from L/min on nasal cannula to ;however, the actual concentration of inspired oxygen inhaled by patients breathing on nasal cannula is known to vary slightly.49 Furthermore, we did not collect data on pharmacologic treatments that the subjects may have received, for example, steroids. However, the RECOVERY trial,20 which first showed benefit of steroids in this population was not released on a preprint server until June 22, 2020, so it may not have had a large impact on clinicians caring for our cohort during the first half of 2020. Similarly, it was May 2020 before data on the benefits of remdesivir became available.21 We postulate that clinician or institutional preference for early or late intubation is likely to be independent of how rapidly they adopted the use of new pharmacologic treatments.
We are also limited by missing data. For example, due to missing data, we were only able to calculate modified SOFA score for a subset of the subjects. We acknowledge the potential for residual differences in the non-respiratory severity of the illness. However, the consistency of our results in the subset of the subjects with a modified SOFA score are somewhat reassuring. In addition, we did not have data on death or disability that occurred after hospital discharge. Also, all the subjects in our cohort were eventually intubated. Although it would be interesting to compare outcomes of subjects with similar ROX index scores who were intubated versus managed conservatively, we did not design our study to do this because we felt it would not be possible to adequately match the subjects to control for clinical factors that influenced the decision to intubate.
Conclusions
This multi-center study of subjects with COVID-19 and who received mechanical ventilation found a high mortality rate. Higher ROX index scores before intubation were positively associated with survival. Older age and increasing days from admission to intubation were associated with mortality.
Footnotes
- Correspondence: Michael T Vest DO, Section of Pulmonary and Critical Care Medicine, Christiana Care Healthcare System, 4745 Ogletown-Stanton Road, Newark, DE 19713. E-mail: mvest{at}christianacare.org
Supplementary material related to this paper is available at http://www.rcjournal.com.
This work was supported in part by an Institutional Development Award from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941 (PI: Dr Hicks).
A version of this paper was presented by Dr Vest at Delaware-CTR Junior Investigator online conference held virtually December 16, 2021.
The authors have disclosed no conflicts of interest.
- Copyright © 2022 by Daedalus Enterprises