Research ArticleOriginal Research

Association Between Rating of Respiratory Distress and Vital Signs, Severity of Illness, Intubation, and Mortality in Acutely Ill Subjects

Aiman Tulaimat, Renaud M Gueret, Mary F Wisniewski and Jacob Samuel
Respiratory Care September 2014, 59 (9) 1338-1344; DOI: https://doi.org/10.4187/respcare.02650

Abstract

BACKGROUND: When deciding whether mechanical ventilation is indicated, physicians integrate their findings on physical examination in a gestalt known as respiratory distress. Despite its importance, this gestalt is poorly understood. This study aims to describe the association between the rating of the severity of respiratory distress and vital signs, severity of illness, use of mechanical ventilation, and death. A prospective observational study with 1,134 consecutive subjects with uncertain triage evaluated by a critical care consult team was carried out in a public inner city teaching hospital.

METHODS: After the initial evaluation of each patient, a critical care physician rated the level of respiratory distress. We recorded vital signs, diagnosis, and laboratory results and calculated the Acute Physiology and Chronic Health Evaluation (APACHE) II score. We recorded if mechanical ventilation was initiated by 72 h and if the subject died during the hospitalization.

RESULTS: The most common diagnoses were respiratory illnesses. Higher distress levels were associated with higher breathing frequency (20, 22, 27, and 30 breaths/min, P < .001) and heart rate (96, 101, 109, and 116 beats/min, P < .001) and lower SpO2 (97, 95, 93, and 92%, P < .001). These variables explain only a small portion of the variance of distress. Distress correlated weakly with the APACHE II score (r = 0.22, P = .001). Blood pressure, temperature, Glasgow coma scale score, and laboratory data were unrelated to the levels of distress. However, higher levels of distress correlated with intubation rates (5, 13, 27, and 41%, P < .001). The area under the receiver operating characteristic curve for respiratory distress predicting intubation (0.72) was larger than that for breathing frequency (0.65). Distress was an independent predictor of intubation but not of death.

CONCLUSIONS: A physician's rating of respiratory distress is independently predictive of intubation in 72 h. Vital signs explain only a small proportion of variance in distress; the other observations contributing to a physician's rating of distress must be determined.

Introduction

The most common reason to initiate mechanical ventilation is to reduce a patient's work of breathing.1 Because it is difficult to measure and interpret the work of breathing at the bedside, physicians search for physical signs that indicate an increase in the work of breathing and integrate them into the gestalt commonly known as respiratory distress.1,2

Campbell3 developed a tool to measure respiratory distress in dying patients. It is comprised of breathing frequency, heart rate, restlessness, accessory muscle use, grunting, nasal flaring, and the look of fear. It was validated against dyspnea during exercise in patients with end-stage lung disease. This procedure was reasonable because the relation between the work of breathing and dyspnea is established.4,5 However, this tool is not used to assess acutely ill patients, and physicians continue to rely on a gestalt to rate respiratory distress. The observations that contribute to this rating are unknown.

Textbooks mention 15 signs associated with increased work of breathing. They provide sparse information on detecting these signs and give no guidance on using them in determining the need for mechanical ventilation.6

The definitions of respiratory distress used in clinical trials are not useful for clinicians. These definitions require continuous monitoring, describe distress when it is severe enough to require ventilator support and not lower levels, consider distress present when any of up to 10 conditions are met, and frequently include respiratory distress as one of these conditions.7–11 The reports of clinical trials rarely report how frequently the conditions were checked, which of them were met, or whether gestalt played any role.

Considering the gap in the understanding of respiratory distress, this study aimed to describe some of its clinical correlates. Specifically, this study sought to determine the association between the rating of the severity of respiratory distress and vital signs, severity of illness, and outcomes.

QUICK LOOK

Current knowledge

The diagnosis of respiratory distress is commonly based on physical examination by experienced clinicians and is an important component of the decision to initiate mechanical ventilation. The rating of the severity of respiratory distress by physicians and the relationship of that assessment to outcomes, the use of mechanical ventilation, and death is not well known.

What this paper contributes to our knowledge

A physician's rating of respiratory distress is independently predictive of intubation within 72 h. Vital signs explain only a small proportion of distress's variance. To understand respiratory distress further, the other observations contributing to a physician's rating of it must be determined.

Methods

Design

This was a prospective exploratory study performed at a public hospital in a large metropolitan area. The institutional review board of the John H. Stroger, Jr. Hospital of Cook County approved the study (03-184) and waived signed informed consent. This allowed us to include all subjects who needed triaging, therefore reflecting real experience.

Subjects

We included all of the subjects evaluated by a critical care team for possible admission to the medical ICU who had not yet received mechanical ventilation, vasopressor therapy, or any intervention that required admission to the unit.12–15 We excluded patients with impending intubation because the evaluating physician could have been postponing an inevitable intubation until circumstances were optimal. This criterion allowed us to include subjects with a wide range of diagnoses whose course was uncertain.

Procedure

The medical critical care team evaluated the subjects either in the emergency department or on the medical wards. A critical care physician completed a standardized form on the first encounter. It included the level of respiratory distress (none, slight, moderate, severe, and severe with impending intubation), the Glasgow coma scale (GCS), and a prediction (0–100%) of the likelihood of intubation and of the likelihood of being alive at discharge.

The physicians were aware that the rating of respiratory distress was intended to describe increased work of breathing and should be based on the respiratory system exam while being independent of laboratory data. To maintain the knowledge and compliance of the physicians involved in the study, the protocol was presented twice in divisional meetings, and a research assistant reminded the physicians every shift to complete a study form for every consultation and answered any questions about data collection.

The vital signs (breathing frequency, heart rate, oxygen saturation (SpO2), mean arterial blood pressure, and temperature) at the time of evaluation and the laboratory data at presentation were collected from the medical record. To determine the severity of illness, the APACHE II score was calculated for all subjects because it requires only 12 variables that are usually available in most acutely ill patients.16 The subjects were followed until discharge from the hospital to determine if they died and if mechanical ventilation was instituted in the first 72 h after the consultation.

Analysis

For normally distributed variables, we used the t test to compare 2 independent groups and analysis of variance to compare 4 independent groups. For correlation, we calculated Spearman's coefficient. For homogeneity, we used the chi-square test. Dichotomous variables are reported as percentages and were compared using the Marascuilo procedure.17 Mann-Whitney and Kruskal-Wallis tests were used to compare variables not normally distributed. For multiple comparisons, we used the Bonferroni correction. To determine the proportion of the variance in the level of distress explained by physiologic variables, we entered the breathing frequency, heart rate, and SpO2 in an ordinal logistic regression analysis to calculate the pseudo-R2.

To determine the accuracy of the level of distress in predicting intubation, we calculated the area under the receiver operating characteristic (ROC) curve and compared it to that of other predictors.18 To determine if the level of distress is an independent predictor of intubation, we compared relevant variables between subjects who were intubated and those who were not and entered the significant ones in a logistic regression model. We also used logistic regression to determine if distress is an independent predictor of mortality after correcting for intubation and APACHE II scores. To rule out the possibility of association by chance, we performed a falsification analysis that entailed determining if there is association between distress and non-respiratory variables. P values ≤ .05 were considered significant. SPSS 16.0 (SPSS, Chicago, Illinois) was used for analysis.

Results

The inclusion criteria was met in 1,134 subjects. Twelve subjects were excluded because intubation was impending. The subjects' mean age was 51 ± 17 y, and 58% were men. Most were African American (54%) or white (23%). Most consultations were in the emergency department (65%), and the remainder were on the medical wards. Eighteen critical care physicians evaluated subjects for the study.

The most common reason for consultation was a respiratory illness (Table 1). Respiratory distress was not present in 625 subjects (55.1%) and was slight in 297 subjects (26.2%), moderate in 173 subjects (15.3%), and severe in 39 subjects (3.4%). Overall mortality was 10.7%. Mortality among the 137 subjects intubated within 72 h was 39.4%.

View this table:
Table 1.

Organ Dysfunction at the Time of Evaluation by the Critical Care Team

Relation to Variables at Presentation

Breathing frequency and heart rate increased with the level of respiratory distress (Table 2). The breathing frequency was significantly different between all levels of respiratory distress (Bonferroni correction, P < .001). The heart rate was significantly different between all levels of respiratory distress (Bonferroni correction, P < .002) except for the comparison between subjects with moderate and severe distress (Bonferroni correction, P = .47).

View this table:
Table 2.

Comparison of Variables as Presentation Between Groups of Distress

The coefficients for the correlation between distress and breathing frequency, heart rate, and SpO2 were 0.42, 0.25, and −0.25 (all P values < .001). In ordinal logistic regression, the breathing frequency, heart rate, and SpO2 were independently associated with the level of distress (P < .001), and the pseudo-R2 was 0.2.

The predicted likelihood of intubation (mean ± SD of 22 ± 23%, median 10% [interquartile range 5–30%]) correlated with the level of distress (r = 0.37, P < .001) and increased significantly with each level (P < .001). The APACHE II score (13 ± 6) correlated with the level of distress (r = 0.22, P = .001). Temperature, blood pressure, GCS, and calcium and creatinine levels were similar between all levels of respiratory distress (falsification analysis).

Prediction of Outcome

One-hundred thirty-seven (12.1%) patients were intubated in 72 h. Use of mechanical ventilation (noninvasive ventilation or invasive intubation) and mortality increased as the level of distress increased (Table 3).

View this table:
Table 3.

Outcomes Based on the Level of Distress

For predicting intubation, the area under the ROC curve (Fig. 1) for the level of distress was higher than the area for breathing frequency (0.72 vs 0.65, 2-tailed P = .009). In the subgroup of subjects with a predicted likelihood of intubation of < 10% (observed intubation rate of 4.7%), the area under the ROC curve for distress was 0.62 ± 0.06.

Fig. 1.
Fig. 1.

Area under the receiver operating characteristic (ROC) curve for predicting intubation within 72 h of evaluation. Respiratory distress (0.72 ± 0.03, P < .001), likelihood of intubation by physician (0.76 ± 0.02, P = < .001), breathing frequency at the time of evaluation (0.65 ± 0.03, P < .001), and SpO2 at the time of evaluation (0.58 ± 0.03, P = .004). The area under the ROC curve for respiratory distress was significantly larger than that for all other variables except the prediction of intubation.

Fifteen of the involved physicians individually evaluated at least 25 subjects. The areas under the ROC curves for their individual ratings of distress were between 0.60 and 0.89 (all P values < .05) and were higher than their corresponding individual areas for breathing frequency except in 2 physicians.

At the time of consultation, the breathing frequency, heart rate, prediction of intubation, and APACHE II score were higher, and the arterial blood pressure, SpO2, and GCS score were lower in subjects who were later intubated than in subjects who were not. Age, anion gap, and concentrations of blood urea nitrogen, creatinine, hemoglobin, and lactic acid were similar between the 2 groups (Table 4). In logistic regression, the level of distress was an independent predictor of intubation (Table 5). The area under the ROC curve for the probability of intubation based on the model was 0.80 (95% CI 0.76–0.84).

View this table:
Table 4.

Comparison of Baseline Characteristics Between Subjects Who Were and Were Not Intubated in 72 h

View this table:
Table 5.

Regression Models for Outcomes

For predicting mortality, the areas under the ROC curves for the level of distress and for the APACHE II score were similar (0.64 and 0.68, P = .15). In a logistic regression model that included the level of distress, intubation, and the APACHE II score, the APACHE II score and intubation were independent predictors of mortality (see Table 5).

Discussion

In this study, we described clinical correlates of the level of respiratory distress rated by physicians on their first encounter with acutely ill patients. Breathing frequency, heart rate, and saturation were independently associated with respiratory distress. Distress independently predicted intubation, but not death. These observations might be axiomatic, but they have never been quantified.

Respiratory distress is a sign and must be distinguished from dyspnea, which is a symptom. Both are a manifestation of an increased work of breathing, and Campbell et al19 reported that the correlation coefficient between them is 0.4.

Respiratory distress was better than breathing frequency in predicting mechanical ventilation. The odds ratio for intubation in 72 h was ∼3 in subjects with moderate or severe distress. This is comparable to the odds of remaining on the ventilator in patients whose dyspnea does not improve after ventilator adjustment.20

Hypotension was not associated with distress but was associated with intubation. We could not determine if it led to lung or diaphragm injury, distress, and ultimately intubation or if hypotension and distress were caused by the patient's illness.13,14

Breathing frequency, heart rate, and SpO2 explained only a small part of the variability of distress. This suggests that the physicians relied heavily on other observations such as the subjects' facial expressions and use of accessory muscles.21

Although the areas under the ROC curves for the level of respiratory distress and the APACHE II score in predicting death were similar, they correlated weakly. The weak correlation might be spurious because of the large sample size (less likely with P < .001) or could suggest that respiratory distress is solely a measure of respiratory dysfunction and explains only a portion of the APACHE II score that incorporates dysfunction of other systems. A strong correlation between distress and the APACHE II score would have weakened our findings because it would have meant that distress is a measure of the overall severity of illness and not of respiratory dysfunction only. In fact, the APACHE II score correlated more with the physicians' prediction of survival than with the physicians' prediction of intubation (data not shown). In a parallel manner, Schmidt et al20 observed that dyspnea during mechanical ventilation was unrelated to the severity of physiologic derangements.

The first strength of our study is the sample. It included a heterogeneous consecutive group of subjects on the medical wards or in the emergency department whose conditions warranted consulting a critical care team.22 Such a sample increases the generalizability of our findings. We also maximized the uncertainty of the triage by excluding subjects whose intubation was imminent or who had any other clear indication for admission to the ICU. This allowed us to evaluate the rating of distress in a challenging circumstance.

The second strength of our study is that we performed falsification analysis. Finding an association between distress and an unrelated variable such as the calcium blood level would weaken (or falsify) our main results because it suggests that the main findings could be just chance. Fortunately, we found no association between distress and body temperature, blood pressure, GCS, and calcium and creatinine levels.23 The third strength is that we included 19 subjects with intubation for every variable included in the logistic regression analysis and that the model had a good fit and an acceptable discrimination.

Our study has 4 limitations. First, the assessors were sometimes involved in the care of the subjects beyond triage. We attempted to correct for this bias by including the physicians' own prediction of the likelihood of intubation. We assumed that this prediction corrects for the implicit inclination or decision by the physician for intubating a patient once conditions permitted (environment, test results, further history, availability of supervision, etc). In the subgroup of subjects with the lowest likelihood of intubation, the ability of distress to predict intubation remained significant but weak. Despite including the prediction in the regression model, respiratory distress remained a significant predictor of intubation. The prediction of intubation was poorly calibrated: the predicted likelihood of intubation was 18 ± 20% in subjects not intubated (overestimation of intubation rate) and 44 ± 32% in intubated subjects (underestimation of intubation rate).

Second, we did not measure patient dyspnea or drive. This would have strengthened our findings but would have made the study more difficult to perform.

Third, we did not measure agreement between observers, although a systematic review by Benbassat and Baumal24 reported that agreement in detecting respiratory distress was fair. Campbell25 reported that a graduate nursing student has a high reliability in detecting fear, use of accessory muscles, paradoxical breathing, and nasal flaring during weaning. We also reported preliminary data that, depending on the time gap between evaluations, agreement on the level of distress was fair to moderate.26

Measuring agreement between physicians would have been prudent if we had used a single observer because the results would have been specific to the observer and not generalizable. Because we relied on a large number of observers, the calculated odds ratios are attenuated by disagreement. Using the equation proposed by Thompson27 and assuming that kappa for the level of distress is 0.2–0.4, the true odds ratio for any level of distress predicting intubation is 3.9–6.9.28,29

Fourth, we did not instruct the physicians on rating the level of distress. This would have changed the gestalt and could have changed the treatment of the subject. The variability introduced by our method is representative of real practice and current knowledge. We assume that the physicians involved in the study used signs that they were confident in detecting, interpreting, and integrating into a gestalt.

We would like to confirm and point out a concerning observation. Despite the clinical significance of the respiratory distress gestalt, physicians in training have limited experience with patients in severe respiratory distress.30 Every month, we supervise four 2-resident teams in the medical ICU. If respiratory distress developed before all intubations, each team would observe 3 subjects in distress.

When determining the need of an acutely ill patient for respiratory monitoring or support, physicians rely heavily on a gestalt assessment of respiratory distress because they do not have reliable tools to measure either respiratory distress or work of breathing. Until physicians have such tools, they must trust their gestalt and closely monitor patients with any level of distress.

Footnotes

  • Correspondence and requests for reprints: Aiman Tulaimat MD, Department of Pulmonary and Critical Care Medicine, John H. Stroger, Jr. Hospital of Cook County, 1900 West Polk Street, Room 1404, Chicago, IL 60612. E-mail: atulaimat{at}cookcountyhhs.org.

  • This work received funding from the Division of Pulmonary and Critical Care Medicine. The authors have disclosed no conflicts of interest.

References

  1. 1.↵
    1. Tobin MJ,
    2. Laghi F,
    3. Jubran A
    . Narrative review: ventilator-induced respiratory muscle weakness. Ann Intern Med 2010;153(4):240–245.
    OpenUrlPubMed
  2. 2.↵
    1. Tobin MJ
    1. Laghi F
    . Indications for mechanical ventilation. In: Tobin MJ , editor. Principles and practice of mechanical ventilation, second edition. New York: McGraw-Hill Professional; 2006:129–162.
  3. 3.↵
    1. Campbell ML
    . Psychometric testing of a respiratory distress observation scale. J Palliat Med 2008;11(1):44–50.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Parshall MB,
    2. Schwartzstein RM,
    3. Adams L,
    4. Banzett RB,
    5. Manning HL,
    6. Bourbeau J,
    7. et al
    . An official American Thoracic Society statement: update on the mechanisms, assessment, and management of dyspnea. Am J Respir Crit Care Med 2012;185(4):435–452.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Petrof BJ,
    2. Legaré M,
    3. Goldberg P,
    4. Milic-Emili J,
    5. Gottfried SB
    . Continuous positive airway pressure reduces work of breathing and dyspnea during weaning from mechanical ventilation in severe chronic obstructive pulmonary disease. Am Rev Respir Dis 1990;141(2):281–289.
    OpenUrlPubMed
  6. 6.↵
    1. Tulaimat A,
    2. Patel A,
    3. Shah B,
    4. Littleton SW
    . Is the content of textbooks on the evaluation of a patient in respiratory distress adequate? Respir Care 2012;57(3):404–412.
    OpenUrlAbstract/FREE Full Text
  7. 7.↵
    1. Keenan SP,
    2. Powers C,
    3. McCormack DG,
    4. Block G
    . Noninvasive positive-pressure ventilation for postextubation respiratory distress: a randomized controlled trial. JAMA 2002;287(24):3238–3244.
    OpenUrlCrossRefPubMed
  8. 8.
    1. Koulouri S,
    2. Acherman RJ,
    3. Wong PC,
    4. Chan LS,
    5. Lewis AB
    . Utility of B-type natriuretic peptide in differentiating congestive heart failure from lung disease in pediatric patients with respiratory distress. Pediatr Cardiol 2004;25(4):341–346.
    OpenUrlPubMed
  9. 9.
    1. Ouanes-Besbes L,
    2. Dachraoui F,
    3. Ouanes I,
    4. Bouneb R,
    5. Jalloul F,
    6. Dlala M,
    7. et al
    . NT-proBNP levels at spontaneous breathing trial help in the prediction of post-extubation respiratory distress. Intensive Care Med 2012;38(5):788–795.
    OpenUrlPubMed
  10. 10.
    1. Stiell IG,
    2. Spaite DW,
    3. Field B,
    4. Nesbitt LP,
    5. Munkley D,
    6. Maloney J,
    7. et al
    . Advanced life support for out-of-hospital respiratory distress. N Engl J Med 2007;356(21):2156–2164.
    OpenUrlCrossRefPubMed
  11. 11.↵
    1. Wysocki M,
    2. Tric L,
    3. Wolff MA,
    4. Millet H,
    5. Herman B
    . Noninvasive pressure support ventilation in patients with acute respiratory failure. A randomized comparison with conventional therapy. Chest 1995;107(3):761–768.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Esbenshade AM,
    2. Newman JH,
    3. Lams PM,
    4. Jolles H,
    5. Brigham KL
    . Respiratory failure after endotoxin infusion in sheep: lung mechanics and lung fluid balance. J Appl Physiol 1982;53(4):967–976.
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    1. Clowes GH Jr.,
    2. Zuschneid W,
    3. Dragacevic S,
    4. Turner M
    . The nonspecific pulmonary inflammatory reactions leading to respiratory failure after shock, gangrene, and sepsis. J Trauma 1968;8(5):899–914.
    OpenUrlPubMed
  14. 14.↵
    1. Scharf SM,
    2. Bark H,
    3. Einhorn S,
    4. Tarasiuk A
    . Blood flow to the canine diaphragm during hemorrhagic shock. Am Rev Respir Dis 1986;133(2):205–211.
    OpenUrlPubMed
  15. 15.↵
    1. Hussain SN,
    2. Marcotte JE,
    3. Burnet H,
    4. Collett P,
    5. Roussos C
    . Relationship among EMG and contractile responses of the diaphragm elicited by hypotension. J Appl Physiol 1988;65(2):649–656.
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    1. Knaus WA,
    2. Draper EA,
    3. Wagner DP,
    4. Zimmerman JE
    . APACHE II: a severity of disease classification system. Crit Care Med 1985;13(10):818–829.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Marascuilo LA
    . Large-sample multiple comparisons. Psychol Bull 1966;65(5):280–290.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. DeLong ER,
    2. DeLong DM,
    3. Clarke-Pearson DL
    . Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44(3):837–845.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. Campbell ML,
    2. Templin T,
    3. Walch J
    . A respiratory distress observation scale for patients unable to self-report dyspnea. J Palliat Med 2010;13(3):285–290.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Schmidt M,
    2. Demoule A,
    3. Polito A,
    4. Porchet R,
    5. Aboab J,
    6. Siami S,
    7. et al
    . Dyspnea in mechanically ventilated critically ill patients. Crit Care Med 2011;39(9):2059–2065.
    OpenUrlCrossRefPubMed
  21. 21.↵
    1. Parthasarathy S,
    2. Jubran A,
    3. Laghi F,
    4. Tobin MJ
    . Sternomastoid, rib cage, and expiratory muscle activity during weaning failure. J Appl Physiol 2007;103(1):140–147.
    OpenUrlAbstract/FREE Full Text
  22. 22.↵
    1. Escarce JJ,
    2. Kelley MA
    . Admission source to the medical intensive care unit predicts hospital death independent of APACHE II score. JAMA 1990;264(18):2389–2394.
    OpenUrlCrossRefPubMed
  23. 23.↵
    1. Prasad V,
    2. Jena AB
    . Prespecified falsification end points: can they validate true observational associations? JAMA 2013;309(3):241–242.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Benbassat J,
    2. Baumal R
    . Narrative review: should teaching of the respiratory physical examination be restricted only to signs with proven reliability and validity? J Gen Intern Med 2010;25(8):865–872.
    OpenUrlCrossRefPubMed
  25. 25.↵
    1. Campbell ML
    . Fear and pulmonary stress behaviors to an asphyxial threat across cognitive states. Res Nurs Health 2007;30(6):572–583.
    OpenUrlCrossRefPubMed
  26. 26.↵
    1. Patel A,
    2. Wisniewski M,
    3. Tulaimat A,
    4. Gonzalez-Carmago C,
    5. Khan J,
    6. Monti C,
    7. et al
    . Do physicians agree in their subjective assessment of patients evaluated for admission to the medical intensive care unit (MICU)? Chest 2007;132(4):446b.
    OpenUrl
  27. 27.↵
    1. Thompson WD
    . Kappa and attenuation of the odds ratio. Epidemiology 1990;1(5):357–369.
    OpenUrlPubMed
  28. 28.↵
    1. Quinn MF
    . Relation of observer agreement to accuracy according to a two-receiver signal detection model of diagnosis. Med Decis Making 1989;9(3):196–206.
    OpenUrlAbstract/FREE Full Text
  29. 29.↵
    1. Faraone SV,
    2. Tsuang MT
    . Measuring diagnostic accuracy in the absence of a “gold standard.” Am J Psychiatry 1994;151(5):650–657.
    OpenUrlPubMed
  30. 30.↵
    1. Albert R,
    2. Slutsky A,
    3. Ranieri VM,
    4. Takala J,
    5. Torres A
    1. Takala J
    . Clinical assessment of the acutely unstable patient. In: Albert R, Slutsky A, Ranieri VM, Takala J, Torres A , editors. Clinical critical care medicine, first edition. Philadelphia: Mosby-Elsevier; 2006:187–198.
Back to top

In this issue

 

Print
Download PDF
Article Alerts
Email Article
Citation Tools

Jump to section

Related Articles

Cited By...

Keywords