Abstract
BACKGROUND: COPD exacerbations lead to excessive health care utilization, morbidity, and mortality. The Ottawa COPD Risk Scale (OCRS) was developed to predict short-term serious adverse events (SAEs) among patients in the emergency department (ED) with COPD exacerbations. We assessed the utility of the OCRS, its component elements, and other clinical variables for ED disposition decisions in a United States population.
METHODS: We compared the OCRS and other factors in predicting SAEs among a retrospective cohort of ED patients with COPD exacerbations. We followed subjects for 30 d, and the primary outcome, SAE, was defined as any death, admission to monitored unit, intubation, noninvasive ventilation, major procedure, myocardial infarction, or revisit with hospital admission.
RESULTS: A total of 246 subjects (median 61-y old, 46% male, total admission rate to ward 52%) were included, with 46 (18.7%) experiencing SAEs. Median OCRS scores did not differ significantly between those with and without an SAE (difference: 0 [interquartile range 0–1)]. The OCRS predicted SAEs poorly (Hosmer-Lemeshow goodness of fit [H-L GOF] P ≤ .001, area under the receiver operating characteristic [ROC] curve 0.519). Three variables were significantly related to SAEs in our final model (H-L GOF P = .14, area under the ROC curve 0.808): Charlson comorbidity index (odds ratio [OR] 1.3 [1.1–1.5] per 1-point increase); triage venous PCO2 (OR 1.7 [1.2–2.4] per 10 mm Hg increase); and hospitalization within previous year (OR 9.1 [3.3–24.8]).
CONCLUSIONS: The OCRS did not reliably predict SAEs in our population. We found 3 risk factors that were significantly associated with 30-d SAE in our United States ED population: triage level, Charlson comorbidity index, and hospitalization within the previous year. Further studies are needed to develop generalizable decision tools to improve safety and resource utilization for this patient population.
- COPD
- emergency care systems
- emergency departments
- emergency department management
- quality improvement
Introduction
Exacerbations of COPD have deleterious effects on quality of life, disease course, and survival.1-5 COPD exacerbations are the leading cause of health care utilization and cost in COPD care.6,7 Since the 2014 inclusion of COPD in the Hospital Readmissions Reduction Program, much effort has focused on reducing in-patient readmissions (http://www.cms.gov, Accessed May 25, 2021).8,9 Optimizing emergency department (ED) care can improve patient-level outcomes and cost of care, as the ED is the gateway for most hospitalizations for COPD exacerbations. In the United States, many EDs have clinical decision units (CDUs) to treat acute conditions for up to 24–36 h and assist in disposition decisions, reduce hospital admissions, and improve resource utilization. Yet, there is little evidence regarding optimal utilization or most appropriate patient population for CDUs in COPD exacerbations.
Among patients with COPD exacerbations presenting to the ED, 37–78% are admitted to the hospital on initial presentation.10-12 Despite high admission rates, almost 50% of adverse events occurs among patients discharged from the ED,13 and those patients discharged home have high 30-d ED revisit rates.14 Specific factors associated with adverse events such as prior severe exacerbations, hospitalizations, intubations, inability to perform a 6-min walk test, and certain vital sign or laboratory abnormalities10,13,15 have been identified in prospective cohorts, but standardized triage processes have not been well validated in clinical practice. It is unclear which subpopulations of patients with COPD exacerbations will benefit the most from observation care in CDUs to mitigate adverse outcomes. The Ottawa COPD Risk Scale (OCRS) was developed prospectively13 and validated16 in several Canadian hospitals for predicting short-term serious adverse events (SAEs), defined as 30-d all-cause mortality or any of the following within 14 d of index ED visit: admission to monitored unit, myocardial infarction (MI), intubation, noninvasive ventilation (NIV), major procedure (coronary artery bypass graft, percutaneous coronary intervention, hemodialysis, or other cardiac surgery), or ED revisit with hospital admission for those discharged on index visit. Many Canadian ED physicians approve of the scale and support its implementation into their practice environment.17 However, the role of the OCRS has not been investigated in the United States health care system with CDU availability.
We assessed the utility of the OCRS, its component elements, and other clinical variables in a United States academic, tertiary-care, urban hospital and determined their performance in disposition decisions to maximize patient safety and minimize unnecessary health resource use.
QUICK LOOK
Current Knowledge
Several factors are associated with adverse events in patients with COPD exacerbations in the emergency department, but standardized triage processes have not been well validated in clinical practice. The Ottawa COPD Risk Scale (OCRS) was developed and validated in Canadian hospitals but has not been investigated in the United States.
What This Paper Contributes to Our Knowledge
The OCRS did not reliably predict severe adverse events in a United States academic, tertiary-care, urban hospital with clinical decision unit capabilities. We identified 3 variables that were associated with significant adverse events in our population, including triage PCO2 level, Charlson comorbidity index, and hospitalization within the past year.
Methods
We performed a retrospective cohort study assessing the utility of the OCRS (Fig. 1) and other predictors in determining short-term adverse events among subjects with COPD exacerbations presenting to the ED. We extracted subject data from our institution’s electronic health record. This study was approved by the University of Cincinnati Institutional Review Board (study # 2017–4084). Patient or the public was not involved in the design, conduct, reporting, or dissemination plans of our research. This study occurred at the University of Cincinnati Medical Center, an urban, academic, tertiary-care center seeing approximately 75,000 ED patients annually, of whom approximately 700 are diagnosed with COPD exacerbations.
Subject records were screened in a stepwise approach for inclusion. All patient records with an ICD-10 code of COPD or emphysema (J40, J41.0, J41.1, J41.8, J42, J43.0, J43.1, J43.2, J43.8, J43.9, J44.0, J44.1, J44.9, J47.1, J47.9, J96.0, J96.9, J80, J96.2, J20.0, J20.1, J20.3, J20.4, J20.5, J20.6, J20.7, J20.8, J20.9, and R09.2) who presented to the ED with any respiratory-related complaint were reviewed for the period of January 1, 2017, to May 31, 2017. Patients were then screened for inclusion and exclusion criteria through manual chart reviews. All charts were reviewed by one reviewer (MD), and 10% of reviews was validated by another independent reviewer (TL) for accuracy and quality. The initial screening produced 873 total records. After further review, 246 were analyzed based on prespecified inclusion criteria. The participant selection was based on criteria similar to the original OCRS derivation study13 that excluded patients who were critically ill upon arrival to the ED. Inclusion criteria included ED diagnosis of COPD exacerbation, a previous diagnosis of COPD confirmed by any prior pulmonary function tests (PFTs) (with FEV1/FVC < lower limit of normal) or emphysema on computed tomographic (CT) imaging, and age > 50 y. Patients were excluded if they required endotracheal intubation or new noninvasive ventilation in the ED, had a resting oxygen saturation < 85% on room air or home oxygen prescription, systolic blood pressure < 85 mm Hg, resting heart rate > 130 beats/min, confusion or severe dementia, ischemic chest pain or acute coronary syndrome, or arrival from skilled nursing facility.
Predictor variables included demographic and historical data, vital signs at presentation, laboratory and imaging study results, electrocardiograms, and time to corticosteroid administration. OCRS scores were calculated for each encounter (Fig. 1). Walk tests were not performed on all subjects—those with heart rates ≥ 120 beats/min or PO2 < 90% after ED treatment were deemed too ill to perform a walk test as surrogate for this original OCRS component.
The primary outcome was a composite of short-term SAEs. These included 30-d ED revisits or in-patient hospitalizations, 30-d mortality, admission to ICU at the index visit, and acute MI or major procedures (coronary artery revascularization, percutaneous coronary intervention, or new hemodialysis) within 30 d. Although the period for rehospitalization in the OCRS derivation study was 14 d, we assessed all outcomes for 30 d as this is more sensitive in detecting ED revisits and hospital readmissions and is the more prevalent outcome duration in the United States as it relates to the Hospital Readmissions Reduction Program.18,19 Data were entered and stored securely in a REDCap database.20
We calculated the OCRS for each subject encounter using the scale’s original components.13 We compared the expected probabilities for an SAE (ie, the predicted risk from the OCRS) to the actual proportion of adverse events across risk categories using chi-square. In addition, we used the OCRS model coefficients applied to our data to assess fit using the Hosmer-Lemeshow goodness-of-fit (H-L GOF) test and the calibration belt21 and discrimination using the area under the receiver operating characteristic (ROC) curve. We conducted a sensitivity analysis by assuming all cases with missing laboratory values were normal for the dichotomous OCRS variables. We also analyzed OCRS and various clinical characteristics using median regression to assess their utility.
To account for potential bias from excluding cases with missing data, we used multiple imputation using chained equations to impute values for missing variables. We generated 10 multiple imputation data sets using the following models to impute missing data: logistic regression for all binary variables; linear regression for hemoglobin, whole-blood , and serum bicarbonate concentrations; predictive mean matching for body mass index, urea, and creatinine concentrations; and Poisson regression for the Charlson comorbidity index. Estimates for all data sets were averaged using Rubin rules. Using the data sets, univariate and multivariable logistic regression were used to estimate the strength of association between all OCRS variables and the outcome, SAE, as a sensitivity analysis for the complete case analyses.
In addition, we generated a final predictive model for the outcome, SAE, using original OCRS variables (continuous or categorical versions) from the development OCRS model as well as additional clinical and demographic variables. To reduce the chance of bias due to excluding cases with missing data, we used the multiple imputation data sets to perform purposeful backward stepwise elimination, with Wald P values used to assess statistical significance for covariate inclusion. All covariates with a P value ≤ .10 for the univariate association with SAE were included in an initial multivariable model, and then variables with P > .05 were excluded one at a time (covariates with highest P value were eliminated first) until only variables with a P value of ≤ .05 remained. All continuous variables were tested for linear association in the log-odds scale (logit) using fractional polynomial regression. To assess model fit and calibration, we used logistic regression for complete cases with the covariates from the final multiple imputation model. We also estimated the area under the ROC curve for the final multiple imputation models, and the model with the highest area under the ROC curve was included as the final model.
Results
Of 873 patients screened, 246 met inclusion criteria. Of those, 191 were complete cases and 55 had incomplete data (Fig. 2). The median age was 61 y, and most subjects were female (54.5%), Black (70.7%), and admitted (51.8%). We identified 46 (18.7%) SAEs overall (1 MI, 4 new administrations of NIV, and 41 admissions/readmissions within 30 d) (Table 1). No subjects died or required intubation within 30 d. For the complete cases in the primary analysis (n = 191), there were 38 (19.9%) SAEs (1 acute MI, 4 new administrations of NIV, and 31 hospital admissions/readmissions within 30 d).
The proportion of subjects with an SAE did not differ significantly by ED disposition (P = .4): discharged from ED, 18/94 (19.2%); admitted to ward, 23/104 (22.1%); CDU followed by discharge, 3/23 (13.0%); and CDU followed by admission to ward, 2/25 (8.0%). Overall, the proportion of subjects with SAEs did not differ significantly between those discharged from the ED (19.2%) and those admitted to the ward or to the CDU (18.4%, P > .99). Subjects transferred to the CDU from the ED had a similar prevalence of SAEs (10.4%) compared to those discharged from the ED or admitted to the ward (20.7%, P = .015).
Median OCRS score did differ significantly by ED disposition (P = .008): discharged from ED, median 1 (interquartile range [IQR] 1–2); admitted to ward, 2 (IQR 1–4); CDU followed by discharge, 1 (IQR 1–3); CDU followed by admission to floor, 3 (IQR 1–4). For subjects who went to the CDU, the median OCRS score was significantly higher for those who went on to be admitted compared to those who were discharged (P = .03). The proportion of SAEs did not differ significantly across OCRS scores (Fisher exact P = .31, test for trend P = .54). The proportion of SAEs also did not differ across OCRS risk categories (P = .33): low (OCRS = 0) 0/1 (0%); medium (OCRS = 1 or 2) 23/117 (19.7%); high (OCRS = 3 or 4) 7/48 (14.6%); and very high (OCRS = 5–9) 8/25 (32.0%). Further, median OCRS scores did not differ significantly (difference: 0, IQR 0–1, P > .99) between those with an SAE (median 2, IQR 1–4) and those without (median 2, IQR 1–3). Median OCRS was higher (P = .047) for those requiring NIV within 30 d (median 5, IQR 3–5) versus for those not requiring NIV (median 2, IQR 1–3). Median length of stay (d) for admitted subjects was significantly higher in those with an SAE (median 3.8, IQR 2.2–5.0) compared to those without (median 2.8, IQR 1.7–3.8, P = .048). Stay for admitted subjects was not significantly related to OCRS (P = .40) or OCRS risk category (P = .34).
Overall, the OCRS fit our data poorly (H-L GOF P ≤ .001, calibration belt P < .001) and did not have adequate discrimination for SAEs (area under the ROC curve 0.519 [0.413–0.625]). In fact, all dichotomized variables used for OCRS development and validation were not significantly related to SAEs in a multivariable model for our study data (likelihood ratio test for all variables combined, P > .99, Table 3). Using the continuous versions for all laboratory variables improved the area under the ROC curve to 0.733; however, only hemoglobin and venous were significantly related to SAEs (Table 3). The sensitivity analysis, assuming all missing laboratory variables were normal, was similar to the complete case analysis, with no dichotomous predictor being significantly related to the outcome (Table 3). For the continuous versions of the OCRS variables, the multiple imputation analysis was similar to the complete case analysis, with only hemoglobin and venous significantly related to the outcome (Table 3).
Our logistic regression analysis using the multiple imputation data identified 3 variables that were significantly related to SAEs in a multivariable model: Charlson comorbidity index (OR 1.3 [1.1–1.5] for each 1-point increase), triage (OR 1.7 [1.2–2.4] for each 10 mm Hg increase), previous hospitalization within 1 y (OR 9.1 (3.3–24.8]). Table 4 shows the detailed model for both the multiple imputation data and the complete case sensitivity analysis, and model discrimination (area under the ROC curve) was good for both the multiple imputation data (0.810) and the complete cases (0.808). Based on the complete cases, model fit (H-L GOF P = .14, calibration belt P = .5) was adequate (Fig. 3).
Discussion
The OCRS did not reliably predict SAEs in our ED CDU study population. We identified 3 variables that were associated with SAEs among subjects presenting with COPD exacerbations: Charlson comorbidity index, previous hospitalization within 1 y, and initial triage PCO2. This model had good fit and discrimination with an area under the ROC curve of 0.8. However, the utility of any of these variables will require validation in prospective multicenter studies.
In our context (midwestern United States ED with CDU availability), the OCRS was not associated with the development of SAEs. OCRS was lower for subjects discharged home compared with those individuals who were admitted to the CDU or hospitalized (this was also true for subjects discharged home from the CDU compared to those admitted from the CDU). There was no significant relationship between SAE and ED disposition; however, OCRS was only significantly associated with one individual SAE, the need for NIV within 30 d, which was associated with a higher OCRS score (median increase = 3) compared to those who did not need NIV. OCRS did not significantly correlate with other individual SAEs or a return to the ED or admission within 30 d. This relationship was true for revisits/admissions associated with COPD and revisits/admissions for any cause. However, there were only 4 cases with a need for NIV within 30 d, which limited practical conclusions. Further, admission stay, whereas significantly related to the probability of an SAE, was not significantly related to the OCRS. Thus, it appeared that higher-acuity subjects did have a higher likelihood of an SAE, as might be expected. However, despite the association between OCRS and ED disposition in our study, there was no significant relationship between SAEs and ED disposition (or OCRS). This finding has motivated the development and evaluation of risk stratification strategies because, as Stiell et al16 showed, 48% (65/135) of adverse events occurred in subjects not admitted to the hospital.
Previous evidence to inform disposition decisions by ED physicians has several limitations. Existing guidelines are primarily based on consensus recommendations and not validated.22,23 Prior risk stratification scores are limited due to a focus on predicting mortality or are not well validated for real-time prospective ED utilization.24-26 The OCRS score attempts to better inform ED physicians in real-time using easily obtainable clinical information available at the bedside to predict short-term SAEs. However, when applied to an urban academic ED in the United States, OCRS and its individual components were not associated with SAEs, nor were SAEs associated with ED disposition. In our study, we found a higher rate of SAE of 18.7% compared to 9.5% observed by Stiell et al.14 This difference could be due to the inclusion of a 30-d period for revisits compared to 14 d used by Stiell et al.14
There are multiple possible reasons for the difference in results compared to the OCRS derivation and validation studies.13,16 One reason could be the differences in socioeconomic, cultural, and behavioral factors between several Canadian EDs and an urban, academic tertiary-care center in the United States. Availability of universal health insurance coverage in Canada may mitigate the influence of socioeconomic factors on health-seeking behaviors and outcomes. Behavioral tendencies of ED physicians can also influence the results, as previous research has demonstrated differences in disposition decisions between practice environments.27 Yet, the disposition decision rates in our study (52% ED admission, 12% CDU admission) are similar to those previously demonstrated (37–78%).10-12 Additionally, health care utilization and health-seeking behavior may vary between regions and health systems, which may impact disposition decisions and care delivery, particularly at institutions with observation capabilities. Further, as was the case in our context, the presence of a CDU to care for patient with low-to-moderate acuity COPD exacerbation under observation settings might influence SAE rates and disposition decisions of ED physicians.
A prospective, well-validated risk stratification tool would be clinically useful to assist ED physicians in determining the most appropriate disposition of patients with COPD exacerbations. The OCRS, whereas validated in 6 Canadian EDs, demonstrated limited utility when applied to a different practice environment. Additionally, most of the SAEs identified in our study, similar to the OCRS derivation and validation studies, were return visits with admissions to non-ICU settings. Serious events (death, MI, intubation, ICU admission) were rare. Developing a practical risk stratification tool useful in the ED setting to help identify SAEs in those patients not admitted to the hospital or CDU will provide extreme value in improving safety and health resource utilization. Further multicenter, prospective studies are needed to bolster the evidence around these decisions. It is plausible that risk scores need to be modified based on different contexts that include socioeconomic factors and health-seeking behaviors.
There are several limitations to our study. First, this was a retrospective review at a single urban, academic hospital that may not represent the entire United States ED population. Further, the relatively small sample size limited the statistical power of our analyses, and future research may be warranted to verify our findings. We limited the inclusion criteria to those adult ED subjects diagnosed with COPD exacerbations who have had any prior PFTs or CT findings of emphysema to support this diagnosis. This may limit the applicability of this study to those patients who have not had formal testing and may impact generalizability. Third, as this was a retrospective study design, we identified several missing values that may have contributed to selection bias. However, we utilized sensitivity and multiple imputation analyses to reduce the risk of bias due to excluding cases with missing variables. We utilized venous values instead of arterial values, which have been shown previously not to correlate well particularly for the highest-acuity patients.28 Despite the limitations of this approach, and as this was a retrospective design, we utilized venous values obtained via venipuncture or indwelling angiocatheter to match the current workflow of our ED triage staff and with considerations of efficiency, patient comfort, and practicality in mind. Additionally, our data extraction was limited to the electronic health record used in our hospital, which may limit details of follow-up. There is a chance of missing SAEs if a subject presented to another hospital. We believe this would be an insignificant effect based on local patient factors and safety-net status of our hospital in the region. Demographic and outcome variables were abstracted via chart review, which may lead to reviewer bias. However, we did utilize an independent second review process for a proportion of cases to ensure accuracy and quality. Further, most of the SAEs in our study were readmissions within the 30-d time frame, which may bias our model (ie, giving more weight to variables that predict readmission like the Charlson comorbidity index or previous admissions) toward this particular SAE and limit its ability to predict other serious SAEs. We did match the criteria for SAEs as was done in the derivation13 and validation studies.16
Conclusions
In summary, the OCRS did not reliably predict SAEs in the context of a United States–based urban, academic hospital. We identified 3 risk factors that were significantly associated with 30-d SAE: triage level, Charlson comorbidity index, and prior hospitalization within past year. A model based on these variables may discriminate high-risk patients in the United States ED setting. Further prospective, multicenter studies are needed to develop ED triage decision support tools to improve patient safety and health resource utilization for patients with COPD exacerbations.
Footnotes
- Correspondence: Timothy M Loftus MD MBA, Department of Emergency Medicine, Northwestern University Feinberg School of Medicine, 211 E. Ontario Street Suite 200, Chicago, IL. E-mail: timothy.loftus{at}nm.org
The authors have disclosed no conflicts of interest.
This study was performed at University of Cincinnati Medical Center, Cincinnati, Ohio.
- Copyright © 2022 by Daedalus Enterprises
References
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.