Abstract
BACKGROUND: Pulse oximetry has become the standard of care in emergency medicine, operating rooms, and medical wards for the monitoring of oxygenation, but the use of pulse oximetry for assessment of hemoglobin (Hb) is controversial. The purpose of this study was to compare the accuracy and precision of 2 point-of-care Hb measurement devices, the Pronto-7 and the HemoCue 201+, to laboratory testing.
METHODS: We studied a convenience sample of patients in the emergency department who required a complete blood count. We excluded patients in critical condition or those with elevated methemoglobin, impaired perfusion, or finger deformities. Each subject provided 2 capillary samples for measurement with the HemoCue 201+ and 2 consecutive readings with the Pronto-7. We used Bland-Altman analysis to compare the performance of the point-of-care devices to laboratory measurements. We also determined the diagnostic performance for the detection of anemia by sex (Hb < 11.6 g/dL for females, Hb < 13.8 g/dL for males).
RESULTS: 201 of the 350 subjects enrolled (57%) were female. Mean (SD) age was 50.9 (19.0) y. Complete data were available for 297 (84.9%) of the Pronto-7 readings and 323 (92.3%) of the HemoCue 201+ readings. Mean (SD) laboratory Hb was 13.1 g/dL (2.3). Mean bias (Bland-Altman limits of agreement) for the Pronto-7 was −0.52 g/dL (−3.29 to 2.25), and for the HemoCue 201+ the mean bias was −0.98 g/dL (−3.57 to 1.61). Sensitivity and specificity for diagnosis of anemia were 81.6% (95% CI 72.5–88.7) and 75.4% (95% CI 68.8–81.1) for the Pronto-7 and 99.1% (95% CI 94.8–100.0) and 71.0% (95% CI 64.4–76.9) for HemoCue 201+.
CONCLUSION: Both devices provided clinically useful methods to screen for anemia.
Introduction
Anemia affects 32.9% of the global population and is associated with nonspecific complaints, which can make diagnosis difficult.1,2 According to the Centers for Disease Control and Prevention, anemia was the primary hospital discharge diagnosis in 188,000 emergency department (ED) visits in 2014.3 Methods for the measurement of hemoglobin (Hb) include testing of venous samples and less invasive point-of-care devices. The latter depend on either capillary blood samples or photometric analysis of blood flow through a fingertip. These devices are typically portable, require little training, and offer potential utility in the prehospital setting or austere environments.4 Advantages of noninvasive devices include reduced pain and discomfort for the patient and reduced exposure to pathogens for health care providers.5
The performance characteristics of the Pronto-7 (Masimo, Irvine, California) and HemoCue 201+ (HemoCue, Ängelholm, Sweden) point-of-care devices, both of which have been cleared by the FDA for noninvasive total Hb spot-check, were compared to venous blood Hb measurements analyzed in a central laboratory (laboratory Hb). Both the Pronto-7 and the HemoCue 201+ have been tested in controlled settings, including blood donation6–8 and out-patient clinics,9–11 but their utility in the ED has not been determined. This study aimed to test the performance characteristics of these platforms in an emergency setting.
The HemoCue 201+ is an absorption spectrometer for Hb measurement that has been in wide use, including pre-donation testing for the American Red Cross, since 2006.12 The HemoCue 201+ requires a small volume of blood (10 mL) taken from a finger using a lancet to activate an azide-methemoglobin reaction in a single-use cuvette. It then uses photometry to measure absorption at 2 wavelengths of light to quantify Hb and to account for turbidity of the sample.13 Hb measurements obtained with the HemoCue 201+ are denoted as capillary Hb.
The Pronto-7 pulse oximeter is a noninvasive device that measures Hb across the capillary nail bed with a finger sensor. The sensor uses multiple wavelengths of light to measure SpO2, pulse rate, and total Hb. A spectrophotometry detector captures different wavelengths of light as they pass through the finger and converts them into a digital signal. Hb measurements obtained with the Pronto-y are denoted as pulse oximetry Hb.
Multiple studies have compared noninvasive devices for Hb measurement, but results have varied.14 One study in 2012 showed the absolute mean difference was 0.56 g/dL (95% CI 0.41 to 0.69) with an upper agreement limit of 2.94 g/dL (95% CI 2.70–3.19) and a lower agreement limit of −1.84 g/dL (95% CI −2.08 to −1.58]).15 A 2011 study compared the performance of pulse oximetry and HemoCue 201+ for Hb monitoring during surgery and found a bias (SD) of −0.02 (1.39) for pulse oximetry and a bias (SD) of −0.17 (1.05) for the HemoCue 201+.13 Because that study was conducted on monitored subjects under anesthesia, relatively few extreme Hb levels were encountered, which could limit the study's generalizability. A review of noninvasive Hb measurements in 2012 suggested the technology had improved for monitoring of hemodynamically stable individuals but identified shortcomings in the presence of peripheral vasoconstriction.16 Previous research has suggested that the HemoCue 201+ is accurate and reliable compared to central laboratory testing.13
We compared the performance of the Pronto-7 and HemoCue 201+ devices to laboratory measurements via Bland-Altman limits of agreement, and we determined the diagnostic performance for the detection of anemia, which defined by laboratory measurement of Hb below the reference laboratory's lower limits for normal Hb levels. These thresholds, based on a normal-range study performed at our institution, were 11.6–15.2 g/dL for females and 13.8–17.3 g/dL for males.
QUICK LOOK
Current knowledge
The use of oximetry for noninvasive assessment of hemoglobin is controversial. Noninvasive point-of-care devices reduce both discomfort for the patient and risk of exposure to pathogens for health care providers. However, the accuracy and precision of this noninvasive device, as well as that of the more invasive systems (which uses capillary blood), in the emergency department is undetermined in comparison to standard laboratory measurement.
What this paper contributes to our knowledge
In a convenience sample of subjects admitted to the emergency department, we found that the Pronto-7 and HemoCue 201+ devices exhibited satisfactory sensitivity and specificity for the detection of anemia. Appropriate applications of these devices in emergency medicine could include use as triage tools, or in patients who would otherwise not receive a blood draw.
Methods
Study Design and Setting
We studied a convenience sample of patients from our ED, which serves approximately 60,000 patients per year and is housed within an academic medical center. All subjects provided written informed consent prior to participation. This study was an investigator-initiated study supported in part by Masimo Corporation. The protocol was approved by the local institutional review board; the full protocol and original research data may be obtained from the corresponding author.
Selection of Participants
Research staff recruited ED patients during periods of peak volume from February 2013 to February 2014. Patients who had an intravenous line and who were scheduled to receive a blood draw for venous Hb testing as part of clinical care were identified via the electronic medical record. Research staff evaluated patient eligibility and obtained informed consent. Both pediatric and adult ED patients were eligible. Patients were excluded if they were clinically unstable, suspected clinically of hemorrhaging, were receiving a transfusion, or were unable to provide informed consent. Patients were also excluded if they were incarcerated, pregnant, intoxicated, or if their fingers would not fit in the Pronto-7 finger probe.
Procedures
Upon determining eligibility, research staff approached potential subjects to obtain consent. ED staff then drew venous and capillary blood for analysis, while research staff obtained measurements with the Pronto-7 device. Two separate blood samples were used for laboratory analysis. A clinical sample was drawn for purposes of patient care, and total Hb from that sample was obtained from the electronic medical record. A second sample was collected separately from each subject for research purposes and was also sent to the hospital's accredited laboratory. Both samples were measured with a hematology analyzer (XN9000, Sysmex, Kobe, Japan). In some cases, blood samples for clinical tests and for research were drawn concurrently. In other cases, the clinical sample was drawn prior to enrollment. In either scenario, 2 laboratory values (ie, the clinical result and the research result) were available for each subject.
While clinical staff collected the blood samples, research staff performed 2 consecutive HbP7 measurements with an appropriately sized reusable sensor (Rainbow 4D sensor, rev F, Masimo) connected to a Pronto-7 device (software version 2317, Masimo). Finger size was measured using the manufacturer's sizing device card to determine the appropriate probe. Results were projected on a display screen if the measurement could be completed. If there was low signal quality during the test, an error message appeared. Capillary samples were obtained with a finger lancet and collected into specialized cuvettes read by the HemoCue 201+. Research staff were trained on the use of both point-of-care devices, including troubleshooting methods in the event one of the devices failed to produce a measurement.
Research staff were blinded to the laboratory Hb results at the time of pulse oximetry Hb and capillary Hb measurement. Subject demographics and laboratory Hb results were obtained from the electronic medical record. Study data were recorded on a case report form and then transferred into REDCap (Research Electronic Data Capture), a secure, web-based application designed to support data capture for research studies.17
Statistical Analysis
Demographic data were summarized with proportions, means, SDs, and 95% CIs. Pulse oximetry Hb and capillary Hb measurements were plotted against laboratory Hb results. After averaging paired data points for each method, a plot of mean versus difference was constructed, and limits of agreement for replicated data were calculated as described by Bland and Altman.18,19 Other measures, including repeatability and confidence intervals for bias and limits of agreement, were reported as recommended by Chhapola et al.20 The coefficient of repeatability is defined by Bland and Altman as the SD of the differences between repeat measurements, all multiplied by 1.96. Its value indicates the range in which the difference between two repeat measurements should fall with 95% probability.19 Sensitivity and specificity were calculated using the thresholds for anemia by sex as determined by a normal-range study performed at our institution (Hb < 11.6 g/dL for females and Hb < 13.8 g/dL for males). We did not perform hypothesis testing to evaluate superiority between the devices because the purpose of this study was to evaluate diagnostic test performance. Analysis was performed with SAS 9.4 (SAS Institute, Cary, North Carolina).
Results
Of the 58,373 patients attending the ED over the study period, 1,531 were screened for eligibility. The most common reasons for exclusion were time constraints and cancellation of a pending hemogram order after placement of an intravenous line (Fig. 1). In our academic medical center, nurses may place intravenous lines and send blood “to hold” pending provider assessment and orders. Patients were often identified as potential subjects during this time frame based on this pending hemogram order. Hemogram orders were frequently cancelled if, after assessing the patient, the clinical provider decided that laboratory testing was not indicated. We considered an enrollment to be complete if it included ≥ 1 laboratory measurement and 2 readings with ≥ 1 of the 2 devices. There were 297 subjects who met this criterion for Pronto-7, and 323 subjects for HemoCue 201+.
Flow chart. ED = emergency department; IV = intravenous line.
Demographic information is provided in Table 1. Of the 350 subjects enrolled, 201 (57%) were female. The subjects ranged in age from 17 y to 100 y old. Mean age (SD) was 50.9 (19.0) y.
Subject Characteristics
Because each of the 350 subjects was scheduled to receive duplicate measurements of each of the 3 methods, there were potentially 2,100 Hb measurements. Mean (SD) Hb for the venous samples (laboratory Hb) was 13.1 (2.3) g/dL. The mean pulse oximetry measurement (pulse oximetry Hb) was 12.6 (1.9) g/dL, while the mean HemoCue 201+ measurement (capillary Hb) was 12.1 (2.5) g/dL.
Despite attempts by research staff to troubleshoot, both platforms occasionally failed to provide a reading due to device error. Of 350 subjects enrolled, 50 subjects (14.3%) had an incomplete set of Pronto-7 readings compared to 22 subjects (6.3%) who had an incomplete pair of HemoCue 201+ readings. Success in obtaining duplicate readings for each subject varied by method: 300 (85.7%) for Pronto-7 and 328 (93.7%) for HemoCue 201+. No subjects refused point-of-care measurement after enrollment. Replicate laboratory testing was performed in 259 (74.0%) of subjects. The most common reason why replicate laboratory testing was not performed was inability of clinical staff to draw the research blood sample from a preexisting line. Only 1 laboratory result was required for inclusion.
The mean (SD) difference between laboratory Hb and pulse oximetry Hb was −0.52 (1.41) g/dL, with the Pronto-7 reading lower than the laboratory value on average. Pulse oximetry Hb values and laboratory Hb values were plotted on a scatter plot (Fig. 2A). Each pair of replicated values was averaged, and the mean and difference between the 2 methods were plotted in another scatter plot (Fig. 3A) with 95% Bland-Altman limits of agreement (−3.29 and 2.25). Confidence intervals for the limits of agreement are in Table 2. The plot does not display any apparent relationship between the magnitudes of the 2 variables, suggesting no need for transformation of the data. The repeatability coefficient was 1.18 for the Pronto-7 across the 297 subjects in this cohort.
Scatter plots of A: Pronto 7 Hb measurements and laboratory Hb measurements, and B: HemoCue 201+ Hb measurements and laboratory Hb measurements. Hb = hemoglobin, R = correlation coefficient.
Bland-Altman plots comparing (A) Pronto 7 values and laboratory values, and (B) HemoCue 201+ values and laboratory values. Center dashed lines show mean difference; red lines denote 95% upper and lower limits, respectively. Hb = hemoglobin.
Correlation and Bland-Altman Analyses of Accuracy of Laboratory Hemoglobin vs Pronto-7 and Hemocue 201+ Devices
For the capillary Hb measurements, the mean (SD) difference between laboratory Hb and capillary Hb was −0.98 (1.32) g/dL, with the HemoCue 201+ measurement being lower on average than the laboratory value. Figure 2B depicts a scatter plot of capillary Hb values versus laboratory Hb values. Each pair of replicated values was averaged, and the mean and difference between the 2 methods were plotted in Figure 3B with 95% (Bland-Altman) limits of agreement (−3.57 and 1.61). Confidence intervals for the limits of agreement are in Table 2. The plot does not display any apparent relationship between the magnitudes of the 2 variables, suggesting no need for transformation of the data. The repeatability coefficient was 2.46 for the HemoCue 201+ across the 323 subjects in this cohort.
We determined the diagnostic accuracy of pulse oximetry for detecting anemia. Subjects were included in this analysis if they had replicate Pronto-7 values and ≥ 1 laboratory measurement. Of 300 subjects with 2 Pronto-7 measurements, 297 had ≥ 1 laboratory measurement. Of these, 80 were true positives for anemia based on the hospital laboratory criteria for anemia as a threshold (Hb < 11.6 g/dL for females and Hb < 13.8 g/dL for males) (Table 3). The Pronto-7 had a sensitivity (95% CI) of 81.6% (72.5–88.7) and a specificity (95% CI) was 75.4% (68.8–81.1).
Accuracy of Pulse Oximetry Measurement by Pronto-7 Device as a Diagnostic Test for Anemia
Of 328 subjects with 2 HemoCue 201+ measurements, 323 had ≥ 1 laboratory measurement. Of these, 105 were true positives for anemia (Table 4). The HemoCue 201+ had a sensitivity (95% CI) of 99.1% (94.8–100.0) and a specificity (95% CI) of 71.0% (64.4–76.9).
Accuracy of Capillary Measurement by HemoCue 201+ Device as a Diagnostic Test for Anemia
Discussion
In this convenience sample, we found that both capillary blood sampling with the HemoCue 201+ and noninvasive Hb measurement via pulse oximetry with the Pronto-7 provided acceptable sensitivity and specificity for clinical use in the detection of anemia. Previous publications have shown similar performance characteristics for the Pronto-7 in different settings. For example, in blood-donor referrals, a comparison of noninvasive and capillary measurements of Hb to laboratory testing of Hb yielded a sensitivity of 63.2% and specificity of 76.2%. That group used a different HemoCue model (ie, HemoCue 301), so their reported sensitivity of 23.1% and specificity of 99.2% for that device may not be comparable.6 Performance characteristics for the HemoCue 201+ as determined by this study differed from previously published studies. One study in 2018 on child anemia in Rwanda found a sensitivity of 89% and a specificity of 86%.21
Lower and upper limits of agreement for both the Pronto-7 (−3.29 and 2.25, respectively) and the HemoCue 201+ (−3.57, 1.61, respectively) were too wide to recommend either device as a full replacement for a blood test because a potential difference of 2–3 g/dL in either direction is significant. However, the demonstrated sensitivity and specificity of either device, combined with their ease of use, means that these devices can potentially be utilized to screen for anemia in patients in the ED setting.
Other investigators have reported an inability to obtain a reading from the Pronto-7, although the degree to which this happened varies widely. We encountered device malfunctions that prevented Hb measurement in 80 out of 700 (11.4%) attempts. Gayat et al5 had only 1 subject out of 300 (0.3%) in the ED setting for whom no reading could be obtained. Sumnig et al6 failed to get readings in 38 out of 515 (7%) blood donors, while Joseph et al22 experienced a much higher frequency of failure in trauma subjects (75 out of 525 subjects, or 14%). Similarly, Khalafallah et al23 evaluated preoperative and oncology subjects and were unable to obtain Hb values in 115 out of 699 (16%) attempts. Anemia was present in 10 of 27 (37.0%) of our subjects in whom a HemoCue 201+ measurement could be obtained but a Pronto-7 measurement could not. This is comparable to the 33.0% (98 of 297) rate of anemia in subjects for whom duplicate Pronto-7 measurements were obtained. Therefore, lack of a reading does not suggest an increased probability of anemia.
Potential limitations of this study include its reliance on convenience sampling and non-randomized screening. Subjects enrolled in this study required a hemogram for clinical purposes and, as such, performance may differ in ED patients who did not require blood testing. Exclusion of patients who were not hemodynamically stable from eligibility may also limit the generalizability of this study to the emergency patient population. In addition, this study used the same reference values to define anemia regardless of subject age. Mean Hb levels have been shown to fall with increasing age.24 While Figure 2 does not display any apparent relationship between the magnitudes of the Hb measurements for either point-of-care device, it is still possible that performance characteristics of these devices could differ in an older study population at those lower concentrations.
Conclusion
The Pronto-7 and HemoCue 201+ point-of-care devices each provide a clinically useful method to screen for anemia in the ED setting. The devices demonstrated acceptable levels of sensitivity and specificity for the detection of anemia, and they could offer utility to emergency medicine in prehospital or austere conditions, as triage tools, or for use with patients who would otherwise not receive a blood draw. Because noninvasive measurement by pulse oximetry provides the additional benefits of patient comfort and reduced health care provider exposure to biohazards such as sharps injury and blood-borne pathogens, pulse oximetry should be considered as a noninvasive alternative to capillary blood measurements for clinical Hb screening.
Footnotes
- Correspondence: Kalev Freeman MD PhD, The University of Vermont, 89 Beaumont Avenue, Given Building, Room E301, Burlington, VT 05405. E-mail: kalev.freeman{at}uvm.edu.
The authors have disclosed a relationship with Masimo Corporation.
See the Related Editorial on Page 1450
- Copyright © 2019 by Daedalus Enterprises
