Article Text
PDF
Abstract
Introduction Adverse events (AEs) affect 3–12% of hospitalised patients. These are estimates from a labour-intensive chart review process,which is not feasible outside research. Clinical deterioration on the wards triggers a rapid response teams (RRTs) consult and can be used to identify an AE prospectively.
Objectives To demonstrate the feasibility of using RRT to detect AEs and compare this methodology to the rates reported using an electronic safety reporting system.
Methods Prospective observational cohort of RRT consults. Three independent physicians reviewed all cases for the occurrence of an AE and its preventability. We summarise AEs as rates per 1000 patient-days, and compared the rates between RRT and the safety reporting system using a Poisson model.
Results There were 8713 hospital admissions, with 531 RRT consults and 247 (2.8%) cases included. Forty-four (17.8%) and 35 cases (14.2%) were judged as AEs and preventable AEs, respectively. RRT identified 0.52 AE/1000 patient-days, compared with 0.21 AE/1000 patient-days detected through the electronic safety reporting system (rate ratio 2.4, 95% CI 1.4 to 4.2, p=0.0014). Patients in surgical wards had more AEs (0.83/1000 vs 0.36/1000, p<0.01) and preventable AEs (0.70 vs 0.21, p<0.01) than patients in medical wards. Agreement for AE (κ 0.46, 95% CI 0.39 to 0.53) and preventable AE (κ 0.47, 95% CI 0.40 to 0.53) was moderate among reviewers.
Conclusions Reviewing RRT consults identified a high proportion of AEs and preventable AEs. This methodology detected twice as many AEs as the hospital's safety reporting system. RRT clinicians provide a complementary and more sensitive mechanism than traditional safety reporting systems to identify possible AEs in hospitals.
- Chart review methodologies
- Adverse events, epidemiology and detection
- Medical emergency team
- Human error
- Incident reporting
Statistics from Altmetric.com
- Chart review methodologies
- Adverse events, epidemiology and detection
- Medical emergency team
- Human error
- Incident reporting
Introduction
Adverse events (AEs) are medical errors that cause harm.1–4 They affect approximately 3–12% of hospitalised patients, of which 33–50% are considered preventable1 These estimates come from large national chart reviews, in which nurses first looked for ‘flags’ of potential AEs (eg, death or unplanned admission to an intensive care unit (ICU)) and physician reviewers then determined the presence of any adverse patient outcomes primarily due to medical care.1–3 ,5–8 Studies using direct observation or more active forms of surveillance have found even higher rates of AEs.9 ,10 All these detection methods require substantial investments of time and money. Moreover, especially in the case of chart review, missing information limits the ability of reviewers to identify AEs or judge preventability. Furthermore, it is clear that existing systems that rely on healthcare worker reporting detect sixfold fewer AEs than active surveillance and miss important AEs.11 Therefore, an efficient method to identify AEs with high specificity and sufficient clinical detail to guide assessments of preventability, and which does not require substantial resources, would be a powerful quality improvement tool for hospitals.
Rapid response teams (RRTs) promote early assessment and intervention for patients with signs of clinical deterioration, aiming to provide early treatment and prevent cardiopulmonary arrest. While the evidence for their efficacy remains conflicting,12 ,13 healthcare practitioners in participating hospitals have reported that RRTs identify specific recurrent preventable AEs that are suitable targets for quality improvement efforts.14
RRT activations were recently used to retrospectively15 or prospectively trigger chart reviews to identify AEs.4 This approach identified 25% of RRT activation calls to be related to preventable AEs, compared with 3–8.5% detection of preventable AEs in other studies using different methods.1–3 ,5–8 ,16 Our primary objectives are to demonstrate the feasibility of using RRT to detect AEs, and to compare this methodology to the rates reported using an electronic safety reporting system.
Methods
We conducted a prospective observational cohort of all RRT consults during a 7-month period, from October 2010 to May 2011 in a tertiary academic centre in Sunnybrook Health Sciences Centre. Because this research was deemed to pose minimal risk to participants, the need for individual informed consent was waived.
Setting and patient selection
The study hospital implemented a RRT in 2006. The system includes a 24-h in-house dedicated nurse, a dedicated staff intensivist during the day (7:30–17:00), available by telephone during off-hours, and critical care residents and fellows available 24 h. RRT provides consultation to all inpatient wards. Any ward team member may activate RRT in response to standard, predefined patient triggers17 or any other concern. Direct requests for ICU consultation are also referred to RRT; therefore, all deteriorating hospital inpatients, detected by staff, excluding those in the emergency department, are seen by RRT.
Data collection
Using a standard 30-min slide presentation, one investigator trained all RRT nurses. All RRT physicians received a copy of the slide presentation, which included six cases designed to address all aspects of AE causation and preventability coding. We asked the RRT nurse to voluntarily fill in a case report form (see online supplementary appendix) for all new consults, including patient demographics, RRT trigger, descriptive summary of the case, RRT actions and diagnosis for the event. We did not specify exclusion criteria, but allowed the nurses and physicians to decide not to include patients if (1) workload didn't allow for it or (2) the case seemed to be trivial or an improper RRT activation. We also asked the RRT nurses and physicians to answer a series of seven questions designed to inform causation by healthcare delivery and preventability, adapted from a previous study,1 and to rate clinical encounters on causation and preventability of AEs (see below). All data were entered into an electronic database (Microsoft Access 2010, Microsoft Corporation) and sent to a panel of three independent reviewers, who were blinded to the RRT assessment of causation and preventability. The reviewers included one intensivist, one surgeon and one emergency medicine physician. All reviewers had at least 5 years of independent clinical practice and independently reviewed all cases. Reviewers were asked to rate clinical encounters in a manner similar to RRT. Reviewers also received a random sample of cases in duplicate, after a minimum interval of 2 weeks, to assess intrarater reliability.
During the same period, we retrieved all major safety incidents reported through the hospital's electronic safety reporting system. Using the hospital's electronic bed management system, we calculated the number of patient-days on the wards during the same period.
Variables and definitions
Our main outcome of interest was AEs occurring on the ward, defined as any unplanned injury arising as a direct consequence of healthcare leading to increased morbidity or mortality and unexplainable by the patient's underlying condition.1 ,18 We further characterised morbidity as (1) requirement for new treatments, (2) prolongation of hospital stay or (3) disability at hospital discharge. We defined preventability as an AE that could have been avoided if errors of omission or commission did not occur.
To ascertain the presence of an AE and its preventability, the RRT nurse and the three reviewers independently judged the patient's condition at the time of RRT assessment using a 6-point Likert scale for causation of an AE and a 6-point Likert scale for preventability of an AE.18 For the Likert scales 1 denoted no evidence of causation (or preventability) and 6 denoted certain causation (or preventability). We classified events with a rating of 4 or higher on each scale as AE or preventable AE.18 For the primary analyses we considered the average rating from the three reviewers.1 After the study was completed, the three reviewers met and categorised all preventable AEs into errors of omission or commission, and into diagnostic or therapeutic errors, achieving consensus on classifications. We allowed preventable AEs to be classified into more than one category.
Statistical analysis
We described the baseline characteristics of patients with means (SDs) or medians (IQRs) for continuous data and or proportions for categorical data. We compared categorical data using χ2 test or Fisher's exact test and continuous variables using t tests if normally distributed or Wilcoxon's rank-sum test otherwise. For our primary analysis, we summarised AEs as rates per 1000 patient-days, and compared the detection rates between the safety reporting system and the RRT using a Poisson model that generated rate ratios. To calculate rates we used all admissions to the wards covered by RRT.
We compared the agreement among reviewers, separately for AE determination and preventability, using agreement statistics. We used weighted κ statistics to compare agreement among reviewers and for the comparison between the mean reviewers’ rating and the RRT rating.19 We used Landis and Koch classification of agreement.20 We analysed intrarater reliability using intraclass correlation coefficients.
To assess the utility of the seven questions directed at detecting AEs, we used univariable logistic regression, with AEs or preventable AEs as the dependent variable, and each question as an independent variable in separate analyses.
Estimates of agreement and rate ratios are reported with 95 CIs. We used SAS V.9.3 (Cary, USA) for all analyses and considered two-sided p<0.05 as statistically significant.
Results
During the 7-month period there were 8713 hospital admissions, with 531 RRT consults of which 247 (2.8% of hospital admissions) were voluntarily reported by the team and included in the study. Patients in our cohort had a median age of 68.5 (SD 17.5) years and multiple comorbidities, and 81 (33%) died in hospital (table 1). The two most common reasons for RRT activation were respiratory failure (83, 34%) and other concerns (94, 38%), and 112 (45%) of patients were transferred to the ICU (table 2). Excluded patients had lower rates of ICU admission (26% vs 45%, p<0.0001).
Patient characteristics (n=247)
Reason for RRT consult and RRT actions (n=247)
RRT identified 44 AEs, for a rate of 0.52/1000 patient-days, compared with 18 safety reports submitted during this period, for a rate of 0.21/1000 patient-days (rate ratio 2.4, 95% CI 1.4 to 4.2, p=0.0014). Although there was a small overlap in the types of events detected by safety reports and RRT (opioid overdose, delays in diagnosis), many of the domains (such as failure to start medications again, failure to provide prophylaxis, falls, failure to identify deterioration) were identified only by RRT or by safety reports (tables 3 and 4). RRT detected 35 preventable AEs, for a rate of 0.41/1000 patient-days (table 5). We classified these 35 preventable AEs into 23 errors of omission and 17 of commission (adds up to more than 35 as some errors were considered under both categories) (table 3).
Types of preventable adverse events (35)
Adverse events reported with the electronic reporting system (18)
Adverse events (AEs) and preventable adverse events/1000 patient days
There were differences in rates of AEs and preventable AEs between services. Medical patients had 0.36 preventable AEs/1000 patient days, compared with 0.83 preventable AEs/1000 patient-days in surgical patients (rate ratio 2.3, 95% CI 1.2 to 4.5, p=0.014). We observed similar results for preventable AEs; medical patients had 0.21 and surgical patients 0.70 preventable AEs/1000 patient-days (rate ratio 3.3, 95% CI 1.5 to 7.4, p=0.004) (table 5).
Agreement for AE was moderate among reviewers (κ 0.46, 95% CI 0.39 to 0.53) and between the reviewers’ average and RRT (κ 0.59, 95% CI 0.51 to 0.68). Agreement for preventable AE was moderate among reviewers (κ 0.47, 95% CI 0.40 to 0.53) and between the reviewers’ average and RRT (κ 0.6, 95% CI 0.52 to 0.68). Intraclass correlation on 44 test-retest cases was 0.86, 0.67 and 0.66 for each reviewer on causation, and 0.88, 0.61 and 0.64 for each reviewer on preventability.
Out of the seven questions in the case report form, three were significantly associated with the detection of an AE: (1) Does the timing of events suggest that the event was related to the treatment or lack of it? (OR 16.9, 95% CI 4.9 to 58.2, p<0.0001); (2) Is lack of treatment a recognised cause of this event? (OR 3.3, 95% CI 1.1 to 9.6, p=0.031); and (3) Do you have other suggestions to improve the process of the care for future patients? (OR 3.2, 95% CI 1.1 to 9.7, p=0.04).
Discussion
We found that a strategy to facilitate the identification of AEs and preventable AEs using RRT consults increased the detection of AE 2.4-fold when compared with the hospital's incident reporting system. Most of the AEs identified through RRT did not overlap with the safety reporting system. RRT detected AEs in about a fifth of the RRT activations and we identified 80% of these AEs as preventable. There was at least moderate agreement among reviewers and between the reviewers’ average and RRT ratings.
Our findings support the use of RRT consults as a trigger for possible AEs. Compared with the previous studies that used RRTs to detect AEs we did not perform a retrospective chart review, which is the most time-consuming part of the process, but relied on data collected by clinicians in real time.4 ,15 We also demonstrated that a panel of reviewers and clinicians had moderate agreement on the adjudication of AEs. This is an important finding, as ongoing reviews of all RRT consults by experts as implemented in this study are not sustainable, but facilitated safety reporting by the RRT may be a feasible and sustainable approach for the detection of AEs, process improvement and continued monitoring.
Our study has important limitations. First, it is a single-centre study and findings may not be generalisable to other settings, specifically, those that do not have RRTs, those that have different RRT models and non-academic hospitals. Second, we excluded a significant proportion of RRT consults. While this leads to under-reporting of AEs, our goal was not to estimate the true incidence of AEs, but to demonstrate the feasibility of using RRTs to facilitate AE detection and compare with the electronic safety reporting system. Furthermore, our study population was more likely to be admitted to an ICU than those that were not included in the study, suggesting that RRT was selective in excluding patients that did not require higher levels of care. Given that we are using the total number of hospitalisation days, including any AEs from the excluded population would further increase the rate of AE detected by RRT. Third, reviewers and RRT staff have the benefit of hindsight bias; knowledge of the outcome influences reviewers’ perceptions of quality of care21 and may have biased our results. However, this is an inherent problem in quality assessment and cannot be mitigated. Fourth, by design we could not detect AEs that do not trigger an RRT consult, nor could we directly compare specific cases between methods. For example, nosocomial infections that do not lead to physiological derangements, falls and cardiac arrest would not be included in our data set. These events are important and can lead to further diagnostic testing, antimicrobial utilisation, prolonged hospitalisation and death. Complimentary methods, such as safety reporting system, code blue logs or discharge codes, are required to include these events.
Because safety-reporting systems are notoriously underused11 ,22 and chart reviews by teams of clinicians for all admissions are impractical, our study suggests that RRT consults may be helpful in detecting AEs and preventable AEs. Although it was possible to have reviewers rate all cases in this study, the additional workload would make this model unsustainable. Therefore our study adds to the current literature in showing that RRT clinicians may be able to facilitate safety reporting by providing a brief case description and answering questions related to AEs.
Conclusion
Reviewing RRT consults identified a high proportion of AEs and preventable AEs. This methodology detected twice as many AEs as the hospital's safety reporting system. It may be possible for RRT clinicians to enter safety reports directly after a consult if they identify a possible AE.
Acknowledgments
The authors thank all the RRT nurses and physicians who contributed to this study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Footnotes
Correction notice This article has been corrected since it was published Online First. Middle initials have been added for two of the authors.
Contributors Grant funding: all authors; study design: all authors; data collection: ACK-BA, NGC, AM; data analysis: ACK-BA and WX; data interpretation: all authors; manuscript writing: all authors.
Competing interests ACK-BA, MC, RAF and NKJA are members of the RRT.
Ethics approval Research Ethics Board at Sunnybrook Health Sciences Centre.
Provenance and peer review Not commissioned; externally peer reviewed.