An interesting study by Tulaimat and Mokhlesi1 regarding the accuracy and reliability of extubation decisions that recently appeared in the Journal merits additional comment. The implicit study question is whether an informed decision to extubate following a successful spontaneous breathing trial is any better than random chance. By study design, the clinical vignettes were selected so that, if a decision to extubate was made by coin flip, without any clinical information, the sensitivity and specificity (as defined in the study) would be expected to reach 50%. It was disappointing that, overall, experienced clinicians performed marginally better than a coin flip in predicting extubation success (ie, 57% sensitivity), but they were highly inaccurate in predicting weaning failure (ie, 31% specificity).
In a post hoc analysis, clinicians whose extubation decision-making was relatively aggressive achieved a higher sensitivity (62%), whereas clinicians whose extubation decision-making was conservative were only able to predict extubation failure with the same accuracy as would occur by chance (52% specificity). Moreover, following a cautious approach was woefully inadequate in predicting extubation success (29% sensitivity). These results seemingly suggest that a relatively aggressive approach to extubation may prevent needless delay in the withdrawal of mechanical ventilation.
The potential impact of a conservative approach to extubation can be appreciated by tallying the additional days of mechanical ventilation that might follow from this decision. At San Francisco General Hospital, in approximately 16% of our medical intensive care unit patients, extubation is delayed following a successful spontaneous breathing trial, because of either altered mental status or pulmonary hygiene problems (2 prevalent factors implicated in extubation failure).2,3 The average duration of additional mechanical ventilation in these patients was 3.5 ± 2.7 days (range 1–13 d). Interestingly, 16% of these patients subsequently experienced substantial deterioration in pulmonary function that may have resulted in failure had a trial extubation occurred. Understandably, this places clinicians in a quandary as to the safe timing of extubation, and likely reinforces a conservative approach.
In that regard, the most intriguing finding of the Tulaimat and Mokhlesi study1 was that when investigators constructed a model to predict extubation outcome (based on the same categories used by clinicians in their decision-making), the model was substantially more accurate than clinicians in predicting extubation outcome. This was demonstrated impressively by the differences in the area under the receiver operating characteristic curve (0.88 vs 0.35, respectively). This suggests that even highly-skilled clinicians have difficulty in successfully organizing and synthesizing information in order to arrive at decisions with a high degree of efficiency. It was interesting for me to discover that the receiver operating characteristic was developed during World War II to assist military personnel in correctly analyzing radar signals to detect enemy aircraft, and has been used extensively in research for over 50 years.4
This raises an important question as to what is hindering the introduction into clinical practice of statistical tools that may improve our clinical decision-making. This is crucial given the larger socioeconomic forces now at play mandating improved healthcare outcomes! Unfortunately, these issues can no longer be relegated to mere academic speculation, but require concrete solutions. In that regard, I would be indebted to both the authors and the editorialists5 if they shared their thoughts on how we should move forward to solve this vexing problem.
- © 2012 by Daedalus Enterprises Inc.
References
The authors respond:
328 329We thank Richard Kallet for his comments on our paper published in this Journal.1 To extubate a patient, the physician must determine if the patient has a sustainable breathing pattern that is adequate for gas exchange, and if the patient can maintain an open airway (or to use the vaguer term, protect the airway).2 Computer-driven weaning is probably an incremental improvement on our knowledge and on our current practice of assessing the breathing pattern that incorporates frequency-to-tidal volume ratio, breathing trial, and weaning protocols.3
On the other hand, keeping an airway open is poorly understood. One reason for this is that we do not have a clear working definition of this concept. An upper airway that is anatomically intact needs 2 functions to stay open: swallowing and the contraction of upper airway muscles during inspiration. Cough is a late line of defense that is essential to manage large amounts of respiratory secretions or secretions that are aspirated because of inadequate swallowing. Most investigators, including us, used combinations of mental status, amount of secretions, and strength of cough as predictors of extubation outcome in patients who tolerate breathing trials. Despite their predictive ability, these variables are surrogates for the 2 functions required to keep an airway open.
The other issue that has not been addressed by research is the sequence of assessing the 2 determinants of successful extubation. During spontaneous breathing without an artificial airway (normal breathing), an open airway is a prerequisite for breathing. This fact is violated in clinical practice. When we evaluate a patient for extubation (a transition to breathing without an artificial airway), we frequently evaluate the breathing pattern first and then decide if the patient can keep the airway open. We believe that this practice is the result of 2 beliefs: the belief that one extra day on the ventilator is more harmful than failed extubation (confirmed by the fact that most physicians in the study were aggressive in extubating patients) and the belief that the available predictors of keeping the airway open are weak (supported by the fact that 21% of the physicians in the study were influenced by mental status and secretions).1 These beliefs might result in the extubation of patients who ultimately required re-intubation because they cannot “protect the airway.” If we perform weaning trials only on patients able to keep the airway open, we will reduce re-intubation from airway related reasons.
Before we replace our current practice with computer aided weaning and extubation algorithms, we should better understand the physiology of the upper airway after extubation and determine the strongest predictors of keeping it open. Then we should determine when to incorporate these predictors, before and after breathing trials. At that point we can use the new knowledge to create comprehensive computer aided algorithms. And even then, gut feeling might remain superior to computers, particularly in complex situations.4
The authors respond:
329 329We agree with Kallet that better tools may help physicians identify extubation candidates more accurately. However, while prediction models constructed along the lines described by Tulaimat and Mokhlesi might prove useful,1 we do not know whether better modeling, in and of itself, would improve decision making. Prediction tools are indispensible when clinicians need to make complex, high risk decisions, but such tools rarely succeed in isolation: they are deeply intertwined with and dependent upon clinical judgment.
An intriguing but unanswered question raised by Tulaimat and Mokhlesi's study1 is why some physicians, but not others, made accurate predictions. As we noted previously,2 the cases studied were difficult, but some physicians clearly got the decision right. All the physicians who participated in Tulaimat and Mokhlesi's study were attendings or fellows in 3 respected teaching institutions1 and presumably familiar with the factors associated with extubation failure. So why did so many choose extubation despite these factors? Did they discount their significance? Did other, unidentified factors persuade them to believe extubation would succeed despite data to the contrary? Did some unnamed bias or heuristic push physicians to make decisions that seem irrational in retrospect? Perhaps identifying the factors associated with accurate decision making would prove illuminating.
A well-constructed algorithm, as Kallet proposes, might improve physicians' ability to predict extubation success. However, the quest to develop such algorithms remains elusive. Despite years of study and countless expert reviews and clinical guidelines,3–6 the pulmonary and critical care community still struggles to find better ways to identify extubation candidates. For example, a recent study by Girault et al showed that a substantial portion (63%) of patients could be successfully extubated despite “failing” a spontaneous breathing trial.7 Clearly, more work is needed to optimize our approach to extubation. Both reliable data—and the optimal use of data available—are vital to the ongoing effort to extubate more of the right patients at the right time.
