Abstract
BACKGROUND: Pulmonary function testing parameters predict cardiovascular and mortality outcomes. Previously, risk scores were created using the basic metabolic profile and complete blood count, including the Intermountain Risk Score (IMRS). This study sought to develop similar pulmonary-specific risk scores for mortality prediction.
METHODS: Subjects evaluated by spirometry at 5 Intermountain Healthcare hospitals (females: n = 2,943; males: n = 2,495) were randomly assigned to risk score derivation (70% of subjects) or an independent validation set (the remaining 30%). Sex-specific scores used spirometry, age, and metabolic and blood count laboratory data. Cox regression β-coefficients formed the basis of risk score weightings.
RESULTS: Among females, pulmonary IMRS was strongly associated with 5-y mortality in the validation set (hazard ratio = 1.24 per +1 risk score, CI 1.16–1.33, P trend < .001), with C-statistics of C = 0.835 and C = 0.757 for derivation and validation, respectively. Among males, validation results were similarly significant (hazard ratio = 1.20 per +1 risk score value, CI 1.11–1.28, P trend < .001), with C = 0.755 and C = 0.699 in derivation and validation sets, respectively. Results were stronger for pulmonary basic metabolic profile risk score, with females having C = 0.815 (derivation) and C = 0.806 (validation), whereas males had C = 0.734 and C = 0.731.
CONCLUSIONS: Pulmonary-specific IMRS and pulmonary-specific basic metabolic profile risk score provided excellent discrimination of mortality among pulmonary subjects. These risk stratification tools combine familiar, relatively inexpensive, commonly-measured, standardized laboratory parameters with spirometry data. They may be electronically calculated and delivered at the point of care, providing meaningful risk information to assist clinicians in patient evaluations.
- red cell distribution width
- RDW
- pulmonary function test
- clinical decision rule
- clinical prediction rule
- Intermountain Risk Score
- IMRS
Introduction
Pulmonary function testing (PFT) is an essential tool for the diagnosis and management of patients with respiratory disease. The most commonly-performed and clinically-useful pulmonary function test is spirometry, which measures the FVC, FEV1, and FEV1/FVC. Although required for diagnosing and staging COPD,1 FVC and FEV1 also predict cardiovascular and all-cause mortality.2,3
Previously, the Intermountain Risk Score (IMRS) was created among general medical patients using the basic metabolic profile (BMP) and the complete blood count (CBC).4 IMRS stratified mortality in other general medical patients, low-risk NHANES (National Health and Nutrition Examination Survey) III participants, and higher-risk coronary angiography patients.4 IMRS research led to the discovery that the red cell distribution width (RDW) predicts mortality.4–6 IMRS also predicts common morbidity end points that lead to mortality, including COPD.50
The finding that IMRS is associated with COPD raised the questions of whether IMRS, its components, or a re-derivation of IMRS that integrated PFT variables could predict mortality in a pulmonary population. Given the lesser ability of IMRS to predict mortality in coronary angiography patients compared with general medical patients,4 it is expected that a risk score integrating PFT-specific data will better predict risk. To determine whether the PFT data elements and re-derived risk values for the CBC and BMP predict mortality in a pulmonary disease population of individuals undergoing spirometry, this study created and tested new pulmonary-specific Intermountain Risk Scores (pIMRS) for mortality.
QUICK LOOK
Current knowledge
Pulmonary function testing (PFT) parameters predict cardiovascular and mortality outcomes in patients with respiratory disease. The Intermountain Risk Score (IMRS) was developed using basic metabolic profile and complete blood count to predict outcomes in general medical patients. The IMRS also predicts common morbidity end points that lead to mortality in chronic lung disease.
What this paper contributes to our knowledge
Pulmonary disease-specific risk scores that employ PFT, the basic metabolic profile, and complete blood count were highly predictive of mortality and provided good discrimination of risk among subjects undergoing PFT. The IMRS was easy to compute using electronic medical records and relatively-inexpensive parameters. This simple tool provided clinically-relevant information in the form of prognostic clinical risk stratification in subjects being evaluated for pulmonary disease.
Methods
Study Population
Subjects evaluated by PFT between October 2002, and October 2011, at 5 urban hospitals in the Salt Lake valley of Utah were considered for inclusion in this study. These hospitals where PFT was routinely performed included Intermountain Medical Center, LDS Hospital, Cottonwood Hospital, Alta View Hospital, and Riverton Hospital. Subjects met inclusion criteria if they were 18 y of age or older and had data for spirometry variables available. Subjects were excluded from the study if their PFT was performed for the purpose of a research protocol; if unique identifying information in the electronic PFT database was missing or incorrect; if measured values were extreme and likely erroneous: forced expiratory time (FET100%) was < 6.0 s (a major data quality indicator)7 or FVC or FEV1 were > 140% of predicted (using NHANES III reference values)8; or if subject age, sex, body mass index, FVC, FEV1, or BMP data were not available. This study was approved by the Intermountain Healthcare Urban Central Region institutional review board (1017618).
Subjects were divided randomly into derivation and validation populations using a long-period Mersenne Twister, with 70% of the subjects (females: n = 2,056; males: n = 1,754) included in the derivation sample. Those 70% were evaluated to derive sex-specific risk scores using the FVC (or FEV1, or FEV1/FVC), body mass index, age, and the CBC and BMP parameters. The other 30% of PFT subjects were held aside as an independent replication set for validating the risk scores (females: n = 887, males: n = 741). Sex-specific modeling was used as per prior evidence in IMRS that showed substantial differences by sex in risk models,4 and because sex-stratification is a standard in the field.9
Study Variables
The first available PFT measurements were utilized as the baseline time point of study entry, with subject age calculated at that time. CBC and BMP laboratory tests performed within 3 months before the PFT or 1 month afterwards were utilized. Missing PFT or other data resulted in the exclusion of subjects with missing data. PFT was performed using the SensorMedics diagnostic system (SensorMedics, Yorba Linda, California), and data were extracted electronically from each machine using the VMax program (CareFusion, San Diego, California). All testing was performed by a certified pulmonary function technician using American Thoracic Society acceptability and repeatability criteria.7 PFT data that were extracted from the PFT results included FVC percent predicted values, FEV1 percent predicted, FEV1/FVC percent predicted, and body mass index. NHANES III reference values for spirometry were used.8
CBC parameters were measured using a clinical laboratory method (Beckman Coulter, Hialeah, Florida). CBC components evaluated were hematocrit, hemoglobin, red blood cell count, mean corpuscular volume, RDW, platelet count, mean platelet volume, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, and total white blood cell count. BMP testing used the Vitros 950 (Ortho Clinical Diagnostics, Rochester, New York) and its components included: sodium, potassium, chloride, bicarbonate, blood urea nitrogen, creatinine, glucose, and calcium. CBC and BMP data were extracted electronically from the Intermountain Healthcare electronic data warehouse, as were subject age and sex. Race data were available for 23% of subjects.
Study Outcomes
The primary end point for which pIMRS was derived was all-cause mortality at up to 5 y of follow-up. Five-year mortality was selected because the majority of spirometry testing is performed among lower-risk out-patients at the 5 hospitals included in the study. Mortality was determined using local Intermountain electronic medical records, Utah State Health Department electronic death certificates, and the national United States Social Security death master file, with deaths recorded through December 2011. Subjects who were reported deceased by one or more sources were considered deceased.
A secondary study outcome was admission to one of Intermountain Healthcare's 22 hospitals in Utah, which was determined by electronic query of a central electronic data warehouse. No risk score was derived for risk of hospital admission, but the mortality risk scores were applied to this outcome to determine whether they also predicted hospital admission.
Statistical Considerations
For risk scoring methods, see the supplementary material (available at http://www.rcjournal.com). The receiver operating characteristic curve was used to determine the C-statistic from pIMRS for both the derivation and validation populations to evaluate and compare the predictive ability in and between both subject sets. Kaplan-Meier survival curves were used to graphically evaluate the association of pIMRS with 5-y all-cause mortality in derivation and validation sets, and the log-rank statistic was computed to evaluate the trend in mortality risk across pIMRS risk strata. Cox regression was used to compute hazard ratio and 95% CI values for the association of pIMRS with 5-y all-cause mortality.
These methods for C-statistic calculations and survival analyses were also used to evaluate 1-y hospital admission after PFT and 1-y mortality. Similarly, the methods were also used to evaluate the association and predictive ability of the pulmonary-specific BMP risk score (pBRS) and the original IMRS with 5-y all-cause mortality in the derivation and validation PFT population sets. IMRS for 5-y mortality was computed as described by Horne et al.4
Descriptive data are summarized as the mean ± SD for continuous variables and frequencies for discrete data. Simple comparisons utilized Pearson chi-square statistic or Student t test. All analyses were performed using SPSS 21 (SPSS, Chicago, Illinois) and used 2-tailed P values with .05 as the threshold for statistical significance.
Results
Females averaged 59.4 ± 15.4 and 58.6 ± 14.9 y of age in the derivation and validation population sets, respectively, and males were 60.2 ± 15.2 and 60.5 ± 15.0 y of age, respectively. Other baseline characteristics are shown in Table 1. PFT and BMP data were available for 2,056 females and 1,754 males in the derivation set, and for 887 females and 741 males in the validation set. CBC data were only available on about half of the subjects 993 females and 847 males in the derivation set; 432 females and 359 males in the validation set). Race was available for 23% of subjects, with the distribution being 1.0% African-American, 2.6% Asian, 0.2% Native American, 3.3% Hispanic, and 92.9% white.
Baseline Characteristics of the Derivation and Validation Groups
Among females, FVC and FEV1 predicted mortality, with a weak association for FEV1/FVC (Table 2). For males, FVC, FEV1, and FEV1/FVC also were associated with mortality risk (Table 2). Because FVC showed the strongest association with mortality in females and the FVC and FEV1 associations were of similar magnitude in males (Fig. 1), FVC was chosen as the primary PFT measure of interest for risk score creation.
The Association in the Derivation Set of FVC, FEV1, and FEV1/FVC With Mortality in Separate Cox Models (Adjusted for the Other Study Covariables)
Kaplan-Meier survival curves for the association of quintiles of FVC (A, C) and FEV1 (B, D) with mortality among females (A, B) and males (C, D), with all P values having P trend < .001.
Risk Score Derivation
For results of the risk score derivations, see the supplementary material (available at http://www.rcjournal.com). The median and range of each risk score are provided in Table 3. Among females in the derivation set, pIMRS and pBRS had C-statistics > 0.80, whereas C-statistics for the risk scores in males were 0.73–0.76 (Table 3). The C-statistics for both pulmonary-specific scores were greater than those for the original IMRS.
Risk Score Results for 5-y All-Cause Mortality
Both risk scores were strongly associated with mortality in females (pIMRS: hazard ratio = 1.51 per +1 risk score value, CI 1.40–1.62, P trend < .001; pBRS: hazard ratio = 1.58 per +1 score, CI 1.48–1.69, P trend < .001). The associations were also strong in males (pIMRS: hazard ratio = 1.33 per +1 score, CI 1.26–1.39, P trend < .001; pBRS: hazard ratio = 1.41, CI 1.33–1.48, P trend < .001).
Validation Population Set
The medians and ranges of the risk scores in the validation set are shown in Table 3, along with the sample sizes and mortality data for females and males. Table 3 also contains the C-statistic data for pIMRS and pBRS. Both risk scores were associated with 5-y mortality in females (pIMRS: hazard ratio = 1.24 per +1 risk score value, CI 1.16–1.33, P trend < .001; pBRS: hazard ratio = 1.48 per +1 risk score value, CI 1.35–1.63, P trend < .001), and in males (pIMRS: hazard ratio = 1.20 per +1 risk score value, CI 1.11–1.28, P trend < .001; pBRS: hazard ratio = 1.34 per +1 risk score value, CI 1.23–1.46, P trend < .001).
Evaluations of mortality using quartiles are shown in survival curves in Figure 2 for pIMRS and Figure 3 for pBRS (for the original IMRS, see Fig. S1 in the supplementary material). For females, the mortality association was hazard ratio = 2.30 per quartile for pIMRS (CI 1.66–3.20, P trend < .001) and hazard ratio = 3.36 per quartile for pBRS (CI 2.35–4.82, P trend < .001). Among males, the quartile associations with mortality were hazard ratio = 2.11 per quartile for pIMRS (CI 1.60–2.79, P trend < .001) and hazard ratio = 2.05 per quartile for pBRS (CI 1.61–2.60, P trend < .001).
Kaplan-Meier survival curves of pIMRS quartiles among A: females (P trend < .001) and B: males (P trend < .001) in the validation set. Quartile ranges were pIMRS ≤ 17, 18–19, 20–22, and ≥ 23 for females, and ≤ 9, 10–12, 13–14, and ≥ 15 for males. pIMRS = pulmonary-specific Intermountain Risk Score.
Kaplan-Meier survival curves of pBRS quartiles among A: females (P trend < .001) and B: males (P trend < .001) in the validation set. Ranges of pBRS quartiles in females were ≤ 8, 9–10, 11–12, and ≥ 13, and for males were ≤ 4, 5–6, 7–8, and ≥ 9. pBRS = pulmonary-specific basic metabolic profile risk score.
High sensitivity (93–99% for females and males) was found for the comparison of quartiles 2–4 to quartile 1 for pBRS and pIMRS, whereas specificity was respectable at 76–80% for both sexes in the comparison of quartile 4 to quartiles 1–3 of either risk score. The strongest result for predictive values, however, was consistently the negative predictive value, which was 97–99% for pBRS and 93–97% for pIMRS among females, whereas males had negative predictive value = 92–99% for pBRS and negative predictive value = 89–95% for pIMRS in the comparison of any combination of quartiles (quartile 4 vs 1–3, 3–4 vs 1–2, or 2–4 vs 1).
For stratified risk score results based on the Global Initiative for Chronic Obstructive Lung Disease (GOLD) stages, see the Supplemental Results and Figures S2 and S3 (available in the supplementary material at http://www.rcjournal.com). Stratified analyses were also performed based on low (< median) and high (≥ median) FVC, which showed better pBRS and pIMRS stratification among those with low FVC than high values (Fig. S4, available in the supplementary material). Finally, results for 1-y mortality and hospital admission end points are provided in the Supplemental Results (available in the supplementary material).
Among females (Fig. 4A), RDW quintile 5 versus 1 was associated with 5-y mortality with hazard ratio = 2.12 (CI 1.13–3.98, P = .020) after full adjustment. For males (Fig. 4B), RDW quintile 5 versus 1 was also associated with mortality (hazard ratio = 2.73, CI 1.59–4.68, P < .001). For further RDW-based analyses, see the supplementary material.
Kaplan-Meier survival curves of RDW quintiles among A: females (P trend < .001) and B: males (P trend < .001) in all subjects (combined population of the derivation and the validation sets). RDW = red cell distribution width.
Discussion
Risk scores using PFT and laboratory variables were highly predictive in both females and males of the risk of future mortality. Among a large population of subjects referred for PFT, pIMRS and pBRS predicted 5-y all-cause mortality. In particular, these results suggest that pBRS provides additional predictive ability beyond the original IMRS, whereas pIMRS provides similarly-excellent results but requires further evaluation.
An abundance of risk scores exists in medicine, and more are being developed, but their implementation in clinical practice is limited to just a few.10,11 Risk scores that predict outcomes for patients with COPD include the APACHE family of risk tools that predict in-hospital mortality12; the BODE (for body mass, airway obstruction, dyspnea, exercise) index, which was derived among only 207 patients to predict mortality during a 2–4-y follow-up13; the simpler ADO index, which has similar prognostic capability when compared with the BODE index14; a COPD exacerbations risk score for predicting mortality15; and a survey-based COPD severity score for predicting disease severity, which was developed to avoid the use of PFT data.16 Unfortunately, each of these risk models requires manual collection and hand-entering of data that are not standard elements in the electronic medical record or requires additional testing usually reserved for higher-acuity patients.
Because financial reimbursement is being tied to hospital and clinician performance and to patients' outcomes, improved but low-cost methods are needed for assessing future risk during each patient evaluation or hospitalization. These methods will predict future patient outcomes well but will need to be easier to compute, less costly, and less resource-intensive than existing tools. The original IMRS and related scores including pIMRS and pBRS were created with these concerns in mind.4,6,17,18 They can be computed automatically inside the electronic medical record and delivered to a clinician at the point of care without changing the care process or involving any clinician in time-consuming data collection or risk score computations. The CBC and BMP data are also standardized, quantitative, objective measures of risk. The availability of robust and easy to obtain risk scores may facilitate clinician acceptance and use. Furthermore, this risk information has a low financial cost because most PFT patients also receive the BMP and CBC laboratory panels routinely.
Further study is required to discover the optimal clinical uses of these risk scores. Some potential implementations can be envisioned, however, due to the high negative predictive value, which suggests that low-risk individuals can be confidently given standard care and that more advanced evaluations may be reserved for the higher-risk patients. The risk scores may be used as quantitative, repeatable, standardized assessments in place of clinical gestalt. For example, clinical application of pBRS and pIMRS may include their use as first-line screening tools to identify higher-risk patients among whom the BODE index or other more expensive, invasive, or time-consuming diagnostic testing or enhanced education or consultations may be used. In this scenario, a risk score is used to identify the high-risk patients so that additional resources and efforts may be used to produce better clinical outcomes, whereas low-risk patients are given standard care (saving both financial costs and clinical time) because applying more extensive testing or additional therapies to low-risk patients will likely have minimal additional benefit. That is, for potential actions that a clinician is considering, the risk score may be used to pursue the action in high-risk patients and to hold off in low-risk individuals when the clinician is deciding whether the action is of value. Computing the risk score multiple times as a patient's care progresses and providing the clinician with data showing risk trends may be particularly useful, which was not possible in this study due to data limitations but was performed in a study of IMRS wherein a second risk computation 6 months to 2 y after the first revealed that both provided independent risk prediction information.17
For clinical interpretation of a new patient's risk score, 3 historical data elements are required to place it in context: (1) the distribution of scores in a historical population, and the risk level for patients of similar age and sex who: (2) had the same score, and (3) had the lowest-risk values for each CBC, BMP, and PFT parameter. Comparing the score to such historical data aids in determining the intensity of additional clinical actions to take. Importantly, historical data are most applicable when they arose from a local population.
Finally, whether pIMRS versus pBRS is chosen for use also depends on whether CBC data are available and other considerations. Because the scores' derivation populations were different, as the mortality rates demonstrate, the selection of pIMRS or pBRS also depends on whether CBC data were collected as a routine care decision or to obtain pIMRS. The existence of CBC data may indicate greater pulmonary disease severity, an in-patient setting, or more comorbidities. The generalizability of pIMRS to those who do not receive CBC testing routinely is unknown, thus additional CBC testing simply to obtain pIMRS is not recommended currently.
Limitations
This study potentially includes the limitations of all observational studies, such as not measuring all important covariables and the inability to completely remove confounding in complex statistical models. For example, data on specific diagnoses of each subject, their symptoms, socioeconomic variables, and smoking status were not available to the study. The analysis did adjust for a plethora of variables among a large number of study subjects using Cox regression; thus, the results are unlikely to represent chance findings. The vast majority of subjects undergoing spirometry at the hospitals included in the study were out-patients, thus the results will apply best to those patients. Due to racial homogeneity of the source population, additional validation of pIMRS in other distinct populations including among minorities is indicated (the original IMRS validated well in external populations).4,18
The source of differences in predictive ability of pIMRS between the derivation and validation populations is not clear. The differences may have resulted from the smaller sample size in which pIMRS was derived, which conveyed a lesser ability to accurately characterize the effect of PFT, CBC, and BMP variables. In contrast, it may reflect that the smaller sample size of the validation population was subject to lower risk estimate precision. Furthermore, the subjects with CBC data were at higher risk of mortality than the full population; thus, pIMRS may have been derived among individuals with more comorbidities that complicate the assessment of risk. Whatever the cause of these differences, further validation of pIMRS is required.
Conclusions
Pulmonary disease-specific risk scores (pIMRS, pBRS) that employ PFT, BMP, and CBC variables–including the RDW–were highly predictive of mortality and provide good discrimination of risk among subjects undergoing PFT. Simple to compute using electronic medical records and employing common, familiar, and relatively-inexpensive parameters, these risk stratification tools provide an additional piece of clinically-relevant information in the form of prognostic clinical risk stratification for use among patients being evaluated for pulmonary disease or respiratory symptoms. Specifically, these tools can seamlessly provide risk information to physicians and other clinicians without requiring those individuals to gather data or compute the scores, likely resulting in greater clinical use.
Footnotes
- Correspondence: Benjamin D Horne PhD MPH, Intermountain Heart Institute, 5121 S Cottonwood Street, Salt Lake City, UT 84107. E-mail: benjamin.horne{at}imail.org.
Supplementary material related to this paper is available at http://www.rcjournal.com.
This research was funded by an unrestricted grant from GlaxoSmithKline. The authors have disclosed no other conflicts of interest.
Dr Horne presented a version of this paper at Chest 2012, held October 20–25, 2012, in Atlanta, Georgia.
- Copyright © 2015 by Daedalus Enterprises