Chest
Volume 112, Issue 2, August 1997, Pages 458-465
Journal home page for Chest

clinical investigations in critical care
Reproducibility of the Histologic Diagnosis of Pneumonia Among a Panel of Four Pathologists: Analysis of a Gold Standard

https://doi.org/10.1378/chest.112.2.458Get rights and content

Study objective

To establish a histologic diagnosis of pneumonia by consensus of a panel of pathologists, to test the interobserver and intraobserver variation in the histologic diagnosis of pneumonia, to compare the diagnostic accuracy of diagnosing pneumonia with and without preselected histologic criteria, and to establish more specific histologic criteria for the diagnosis of pneumonia.

Methods

The study group consisted of 39 patients who died after a mean of 14 days of mechanical ventilation. A postmortem open lung biopsy was performed on all patients. The tissue was reviewed independently by four pathologists who categorized the slides from each patient as showing or not showing pneumonia. Interobserver variation was calculated using the kappa statistic. Six months following the initial evaluation, the same slides were resubmitted to one of the pathologists for reevaluation to look for intraobserver error. Finally, the slides were reviewed and categorized by the criteria of Johanson et al into no pneumonia, mild, moderate, or severe bronchopneumonia. A comparison was made of the patients selected as demonstrating histologic pneumonia by each of the examinations.

Results

The reliability coefficient (kappa) measuring agreement among the four pathologists was good at 0.916. However, the prevalence of pneumonia as determined by each of the four pathologists varied; pathologist A, 15 of 39 (38%); pathologist B, 12 of 39 (31%); pathologist C, 9 of 39 (23%); and pathologist D, 7 of 39 (18%). Resubmitting the same slides to the same pathologist 6 months later resulted in reclassification of 2 of 39 patients. Using the histologic criteria of Johanson and colleagues, 14 patients were selected as having pneumonia compared with only nine patients selected by consensus of three of four pathologists.

Conclusions

Recognition of histologic pneumonia varies among pathologists. The preselected criteria of Johanson and colleagues detected histologic pneumonia in eight of nine patients picked by consensus of pathologists, but six additional patients classified as “no histologic pneumonia” by the consensus of pathologists were judged to have histologic pneumonia by these criteria. The results established the necessity for standardization of histologic criteria for studies using biopsy as the gold standard for bacterial pneumonia. An atlas showing the criteria used in our selection was developed.

Section snippets

Study Design

Prospective, cross-sectional study of 40 patients dying while receiving mechanical ventilation.

Patient Population

Patients were recruited from the combined medical-surgical ICU of a 234-bed teaching hospital. The criteria for inclusion were as follows: (1) the patient died following at least 24 h of mechanical ventilation; (2) the patient was hospitalized for a minimum of 72 h prior to death; and (3) specimen collection could be initiated within 2 h of death. The criteria for exclusion were as follows: (1)

Results

From August 1991 to October 1994, 40 patients were prospectively enrolled. The pathology specimen from one patient was lost in the course of the study and the related data were excluded from analysis. Collection of specimens was initiated a mean 57 min after death (range, 15 to 120 min). The group consisted of 20 women and 19 men with a mean age of 66 years (range, 38 to 85 years). Thirty-six of 39 patients had airspace disease on their most recent chest radiograph and 38 of 39 patients had

Discussion

Validation of histologic pneumonia is the essence of studies correlating microbiological data with postmortem pathologic findings in patients with VAP. Despite this, to our knowledge, no prior study has tested the reproducibility of histologic diagnosis or tried to establish diagnostic references. The tacit presumption has been “the pathologist can recognize pneumonia.” A 1992 consensus conference addressing standardization in study design defined VAP as “a nosocomial, usually bacterial

ACKNOWLEDGMENT

The authors gratefully acknowledge Rae Wu, MA, MPH, for her statistical analysis, and Jane Lopez for her superb assistance in manuscript preparation and editing.

References (14)

There are more references available in the full text version of this article.

Cited by (86)

  • A systematic review classifies sources of bias and variation in diagnostic test accuracy studies

    2013, Journal of Clinical Epidemiology
    Citation Excerpt :

    There was some evidence that interobserver variability is greater than intra-observer variability (one experimental study) [69] and that sensitivity and overall accuracy were higher for experts than for non-experts (one review [54], two DTA studies [67,68], and two experimental studies [71,75]). Two reviews [22,77] and one experimental study [76] found no evidence of association between observer experience and sensitivity/specificity. There was consistent evidence that the availability of clinical information to the person interpreting the test result increased sensitivity and overall accuracy; the effect on specificity varied.

  • Ventilator-associated pneumonia: A review

    2010, European Journal of Internal Medicine
    Citation Excerpt :

    But they are also associated with certain inherent problems. There is a significant variation of about 18% to 38% in interpretation of the histopathological findings by different pathologists [80]. Furthermore, patients included in the postmortem studies may not be truly representative of most patients with VAP and so the evaluation of other diagnostic tools in comparison to these “gold standards” may not be fully justified [2].

  • The autopsy pathology of sepsis-related death

    2007, Current Diagnostic Pathology
View all citing articles on Scopus

This work was funded by the Edward H. Morgan Fund for Clinical Research in Pulmonary Disease, Virginia Mason Research Center, Seattle.

View full text