Research ArticleOriginal Research

Accuracy of Multiple Pulse Oximeters in Stable Critically Ill Patients

Marie-Anne Blanchet, Gabriel Mercier, Antoine Delobel, Emi Nayet, Pierre-Alexandre Bouchard, Serge Simard, Erwan L'Her, Richard D Branson and François Lellouche
Respiratory Care May 2023, 68 (5) 565-574; DOI: https://doi.org/10.4187/respcare.10582

Abstract

BACKGROUND: An accurate SpO2 value is critical in order to optimally titrate oxygen delivery to patients and to follow oxygenation guidelines. Limited prospective data exist on real-world performance of pulse oximeters in critically ill patients. The objective of this study was to assess accuracy and bias of the SpO2 values measured by several oximeters in hospitalized subjects.

METHODS: We included stable adults in the ICU with an arterial catheter in place. Main exclusion criteria were poor SpO2 signal and SpO2 > 96%. In each subject, we simultaneously evaluated 4 oximeters: Nonin (Plymouth, Minnesota) embedded in the FreeO2 device (OxyNov, Québec City, Québec, Canada), Masimo (Radical-7, Masimo, Irvine, California), Philips (FAST, Philips, Amsterdam, the Netherlands), and Nellcor (N-600, Medtronic, Minneapolis, Minnesota). Arterial blood gases were drawn and simultaneously each oximeters' SpO2 values were collected. SpO2 values were compared to the reference (arterial oxygen saturation [SaO2] value) to determine bias and accuracy. The ability for oximeters to detect hypoxemia and the impact of oximeters on oxygen titration were evaluated.

RESULTS: We included 193 subjects (153 male, mean age 66 y) in whom 211 sets of measurements were performed. The skin pigmentation evaluated by Fitzpatrick scale showed 96.2% of subjects were light skin (types 1 and 2). One oximeter overestimated SaO2 (Philips, +0.9%), whereas the 3 others underestimated SaO2 (Nonin −3.1%, Nellcor −0.3%, Masimo −0.2%). SaO2 was underestimated with Nonin oximeter in 91.3% of the cases, whereas it was overestimated in 55.2% of the cases with Philips oximeter. Moderate hypoxemia (SaO2 86–90% or PaO2 55–60 mm Hg) was detected in 92, 33, 42, and 11% of the cases with Nonin, Nellcor, Masimo, and Philips, respectively.

CONCLUSIONS: We found significant bias and moderate accuracy between the tested oximeters and the arterial blood gases in the studied population. These discrepancies may have important clinical impact on the detection of hypoxemia and management of oxygen therapy.

Introduction

Pulse oximetry is used daily by clinicians around the world to obtain a noninvasive measurement of SpO2, a reflection of arterial oxygen saturation (SaO2) that enables detection and amelioration of hypoxemia (main objective)1–4 and hyperoxemia, both deleterious states.5–12 Monitoring of pulse oximetry, often considered as the fifth vital sign, is recommended for patients with respiratory failure requiring oxygen therapy.13 Several guidelines for oxygen therapy aim to reduce both the risk of hypoxemia and hyperoxemia.13–15 In order to follow these guidelines, an SpO2 value that accurately represents the SaO2 value is required. It has been shown that small variations in SpO2 target greatly affect the oxygen flow required, which may have a relevant impact on clinical decisions.16–19 Furthermore, SpO2 is frequently used to detect hypoxemia and assess the severity of patients with pneumonia. SpO2 and oxygen support also have an impact on clinical decision-making such as the initiation of corticosteroid treatment, intubation, or transfer to an ICU. For all these reasons, pulse oximeter accuracy is desirable. Although modern pulse oximetry was described 50 years ago, basics of this technology have not evolved significantly. Main evaluations of accuracy of these devices are carried out in healthy volunteers, with the exception of a few studies conducted in subjects.20–24 For certification purpose, studies are conducted in healthy subjects,25 whereas a number of common clinical conditions encountered in patients can reduce pulse oximetry accuracy. The conditions that may alter oximeters' precision include skin pigmentation,20,26–30 skin temperature, skin thickness, nail polish, movement artifacts,20,21,31–33 and poor peripheral perfusion.34,35 The impact of the oximeter brand used on measurement accuracy has been poorly evaluated in subjects with currently available devices.21,22,26,27,31,36

No recent study has simultaneously compared SpO2 values from multiple oximeters to the reference standard SaO2. As a result, the assessment of agreement between SpO2 and SaO2 with currently available oximeters is required. The objective of this study was to describe the accuracy and bias of the SpO2 value measured by several oximeters (Nonin, Nellcor, Masimo, and Philips) frequently used in clinical practice compared to the reference value, SaO2, measured by arterial blood gases in stable subjects admitted in an ICU.

QUICK LOOK

Current knowledge

An accurate SpO2 value is critical in order to optimally titrate oxygen delivery to patients and to follow oxygenation guidelines. Oximeters' accuracy has not been well evaluated in subjects, although real-life conditions may reduce pulse oximetry accuracy in comparison with evaluations conducted in healthy volunteers with hands wrapped in a warming pad.

What this paper contributes to our knowledge

The present study shows that bias and accuracy of oximeters were variable in these medical instruments. We found significant bias and moderate accuracy between the tested oximeters and the arterial blood gases in the studied population. Oximeters are not all similar and provide different information. These discrepancies may have important impact on management of oxygen therapy and on clinical decision-making related to oxygenation.

Methods

This study compared SpO2 values from 4 oximeters to concomitant SaO2 on 193 subjects in the ICU at Institut Universitaire de Cardiologie et de Pneumologie de Québec (IUCPQ), in Québec City, Canada, between March 24, 2021–September 17, 2021. The study was approved by the IUCPQ Research Ethics Committee with a waiver of consent.

To be eligible, adult subjects (age ≥ 18 y) were to be stable in terms of oxygen administration parameters (no change in FIO2 and O2 flow for the last 30 min) and hospitalized in the ICU with an arterial catheter in place. Exclusion criteria were instability requiring frequent ventilator parameter modifications, poor SpO2 signal (based on the pulse oximetry waveform), recent ICU admission (considered unstable), infectious isolation, SpO2 > 96%, methemoglobinemia > 1.5%, and presence of dark nail polish.33 Subjects could not be studied more than once per day.

The evaluated oximeters were Nonin (Plymouth, Minnesota) embedded in the FreeO2 device (OxyNov, Québec City, Québec, Canada), Masimo (non–touch screen Radical-7, Masimo, Irvine, California), Philips (FAST, Amsterdam, the Netherlands), and Nellcor (N-600, Medtronic, Minneapolis, Minnesota). The slowest moving average time for SpO2 display was set on each of the oximeters (Nonin, 4 beats; Nellcor, 5–7 s; Masimo, 4 s; Philips, 5 s). Reusable digital sensors for all oximeters were used.

Before scheduled arterial blood gases, the oximeter sensors were placed on fingers in random order.32 The fingers were randomized for the 4 tested oximeters (Nonin, Nellcor, Masimo, and Philips). The fingers used were the index, middle, and ring fingers of each hand.

Once all oximeters were in place, 2 min passed before the measurements were taken to ensure stability of the values. We used the pulse oximetry waveform to determine a signal quality for all oximeters. Four different technicians evaluated the pulse oximetry waveform for the included subjects. They all required mutual agreement on the quality of waveform. Then, each oximeter's SpO2 values were collected simultaneously to the arterial blood gas. SaO2 was used as the reference standard since it is commonly used as the reference in multiple previous studies (functional saturation was reported as recommended).20–24 It was determined by multiwavelength oximetry (ABL800 FLEX, Radiometer, Copenhagen, Denmark).

The primary outcome measurements were SpO2 values for each evaluated oximeter and SaO2 values from the arterial blood gases. Subjects' vital signs (heart rate, breathing frequency, blood pressure, temperature, and oxygen saturation at inclusion) were collected, as well as cardiac index if available within the last 2 h for subjects after cardiac surgery. Skin color was determined using the Fitzpatrick scale;37 comorbidities such as diabetes mellitus, presence of peripheral vascular disease, and left heart failure (defined by left ventricular ejection fraction < 40%) were all collected, as well as vasopressors or inotrope medication doses and the type of respiratory support. The main laboratory results collected were SaO2, pH, HCO3−, PaCO2, PaO2, lactate, methemoglobinemia, carboxyhemoglobinemia, hemoglobin, and glycosylated hemoglobin.38

Statistical Analysis

From a previous unpublished experiment conducted in patients, with 42 measurements, the mean difference between 2 oximeters was 1.1 ± 1.2%. At a global significance level of 5% and applying the Bonferroni criteria for multiple comparisons, 210 records were required to detect a mean difference of 0.5% using a power of 95%. Using 35 randomization blocks, 210 participants could be enrolled. Continuous variables are reported as mean ± SD or median with interquartile range as appropriate. Nominal variables are reported as frequencies. The primary outcome evaluates the accuracy (random error) and bias (systematic error) of the SpO2 value measured by different oximeters frequently used in the clinical practice compared to the reference value of SaO2 measured by arterial gases in subjects managed in the ICU. The first approach was the use of Bland-Altman plot to analyze the agreement between values of SaO2 measured by arterial gases and values of SpO2 measured by oximeters. We reported mean bias, precision, and 95% CI. Mean bias was calculated as SpO2−SaO2 from each oximeter's value and the corresponding arterial blood value. Precision was calculated as SD of the bias. We reported root mean square error (RMSE), a parameter required by the Food and Drug Administration (FDA) for pulse oximeters' certification. RMSE is derived from calculations involving bias and precision.

Intra-class correlations were reported for assessing agreement between the reference values for SaO2 and pulse oximetry values from oximeters separately. Relationships between the reference values for SaO2 and pulse oximetry values from oximeters were analyzed using Pearson correlation coefficient. The linear mixed model was performed on a Latin-square design using an unrestricted covariance structure to compare oximeters (fixed factor). Blocks were linked to the random effect. The normality assumption was verified with the Shapiro-Wilk tests using residuals from the statistical model and transformed by the Cholesky metric. The Brown and Forsythe variation of Levene test statistic was used to verify the homogeneity of variances. Test was performed at 5% level.

Factors that modified the SpO2−SaO2 gap for each tested oximeter were analyzed using a linear regression model. Multivariable linear regressions were performed to determine independent predictors. Continuous variables were checked for the assumption of linearity. The final model was selected on the smallest Akaike criteria. The statistical analysis was not planned for multiplicity in test for secondary outcomes. All reports estimates are expressed using mean and 95% CI. Some baseline characteristics were used to investigate if any may explain the variability among oximeters by adding interaction terms between oximeters and variables. The hypoxemia detection was analyzed for each oximeter. The areas under the curve were compared using univariable models. Analyses were performed using SAS version 9.4 (SAS Institute, Cary, North Carolina) and R package (R Foundation for Statistical Computing, Vienna, Austria).

Results

We collected 1,055 simultaneous measurements of oxygen saturation (one SaO2 and 4 SpO2 values for 211 inclusions in 193 subjects), 153 inclusions (73%) in males and 58 (27%) in females. Subjects could only be included once a day in the study. The mean age was 66 ± 11 y. The skin pigmentation, using the Fitzpatrick scale, was of type 1 or 2 in 96.2% of subjects. Sixty-four subjects had diabetes; 32 had peripheral vascular disease; 24 had left heart failure; 60 subjects received vasopressors or inotropes at time of measurement; 22 subjects received invasive ventilation (mean FIO2 was 0.37 ± 0.09); 15 received high-flow oxygen therapy; 3 received noninvasive ventilation; 87 received conventional oxygen therapy (mean O2 flow was 2.9 ± 4.8 L/min), and 83 did not have respiratory support (Table 1 and study flow chart Figure S1, see related supplementary materials at http://www.rcjournal.com). Thirty-one subjects had tachycardia (heart rate > 100 beats/min); 15 were hypotensive (systolic blood pressure < 100 mm Hg), and none were hypothermic (core temperature < 35°C).

View this table:
Table 1.

Subject Characteristics and Clinical Features

Mean SaO2 value was 93.6 ± 2.2%. There was positive bias of +0.9% for the Philips oximeter. There was negative bias for the oximeters Nonin, Nellcor, and Masimo, with bias of −3.1, −0.3, and −0.2%, respectively (Table 2). A Bland-Altman plot was drawn for each oximeter to illustrate agreement between SpO2 and SaO2. Mean bias ± precision and 95% CI were −3.1 ± 2.1 (−7.2 to 11.0) for Nonin, −0.3 ± 1.7 (−3.6 to 3.1) for Nellcor, −0.2 ± 1.9 (−4.0 to 3.6) for Masimo, and 0.9 ± 1.9 (−2.9 to 4.7) for Philips (Fig. 1). The Nonin oximeter underestimated SaO2 in 90.3% of SpO2 values, whereas Philips overestimated SaO2 in 55% of the cases (Table 2). Overall, the frequency of differences ≥ plus or minus 4% were present in 35.2, 2.4, 8.6, and 9.5% of the pairs with Nonin, Nellcor, Masimo, and Philips, respectively.22 Bias ≥ plus or minus 10% was present in 1.5% of the pairs with the Nonin oximeter (Table 2). The impact of several clinical and biological parameters on the bias was assessed (peripheral arterial disease, heart failure, type of ventilation, vital signs, diabetes mellitus). There was only a minimal impact for the parameters tested. The impact of skin pigmentation could not be adequately assessed given the low number of subjects with dark-skin pigmentation (Table S1 and S2, see related supplementary materials at http://www.rcjournal.com).

Fig. 1.
Fig. 1.

Representation of the Bland-Altman plot for the comparison of the pulse oximeter saturation displayed by tested oximeters with the reference value (arterial oxygen saturation). A: Nonin, B: Nellcor, C: Masimo, and D: Philips oximeters. Black horizontal lines show the bias, and dashed lines denote 95% CI. SaO2 = arterial oxygen saturation.

View this table:
Table 2.

Main Features of Evaluated Oximeters

In terms of accuracy, the tested oximeters all showed large random errors. The coefficient of determination (r2) for each oximeter expressed moderate correlation between SpO2 values and SaO2 values (Figs. 1 and S2). The Nellcor oximeter had an r2 coefficient that expressed a slightly greater accuracy than the other oximeters, with an r2 value of 0.51. The 3 other oximeters r2 coefficients were, respectively, 0.43, 0.43 and 0.44 for Nonin, Masimo, and Philips. The RMSE for each oximeter was of 3.73, 1.72, 1.95, and 2.13 for Nonin, Nellcor, Masimo, and Philips, respectively.

Oximeters frequently show diverging values for the same measurement. Nonin and Philips oximeters have shown the most significant bias, with mean bias of 4.0% (Table S3, see related supplementary materials at http://www.rcjournal.com). Masimo and Nellcor oximeters have shown concordant results, as the mean bias between these 2 oximeters' SpO2 values was of 0.1% (Table S3). The relationship between oximeters was represented by point clouds with poor determination coefficient (r2), as shown on the Figure S5 (See related supplementary materials at http://www.rcjournal.com).

SaO2 and PaO2 values allow detection of hypoxemia commonly defined as SaO2 < 90% and PaO2 < 60 mm Hg.20 The ability to detect hypoxemia was variable between the tested oximeters. In the included subjects, 19 had a PaO2 value from 55–60 mm Hg, and 12 had an SaO2 value from 86–90%, suggesting moderate hypoxemia. The Nonin oximeter was the most sensitive of the 4 tested oximeters for detection of hypoxemia. It detected 100% of the states of moderate hypoxemia. On the other hand, the Philips oximeter detected only 11% of hypoxemic events. Masimo and Nellcor detected 37% and 26% of hypoxemic events, respectively (Figs. 2, 3, and S4, see related supplementary materials at http://www.rcjournal.com).

Fig. 2.
Fig. 2.

Detection of arterial hypoxemia defined by PaO2 < 60 mm Hg (A: in 19 subjects that met this definition) and SaO2 < 90% (B: in 12 subjects) by the tested oximeters Nonin, Nellcor, Masimo, and Philips.

Fig. 3.
Fig. 3.

SpO2 values of the 4 tested oximeters for SaO2 values < 90% (A) and for PaO2 values < 60 mm Hg (B). SaO2 = arterial oxygen saturation.

Based on the cohort of the 211 concomitant measurements of oxygen saturation, we evaluated for each oximeter if oxygen (or FIO2) should be increased, decreased, or maintained constant to keep the subject within an SpO2 target of 92–96%. With this objective, it would require an increase oxygen in 60.6% versus 19.1% versus 18.7% versus 10.4% and to decrease oxygen in 1.0% versus 6.2% versus 9.1% versus 20.9% of the cases (P < .001) when using the Nonin, Nellcor, Masimo, or Philips oximeter, respectively.

Discussion

Our study showed that in our population of stable subjects hospitalized in ICUs tested pulse oximeters had moderate accuracy (random error). In addition, each oximeter had significant differences for bias (systematic error) when compared to the reference standard, SaO2. The largest bias was shown with Nonin oximeter, with a mean bias of −3.1%, followed by Philips oximeter with a mean bias of +0.9%. Masimo and Nellcor oximeters did not show a significant bias, but, as for all tested oximeters, the correlation between SpO2 and SaO2 showed a moderate accuracy. Indeed, in about 14% of the subjects, the bias between pulse oximeters saturation and SaO2 was ≥ 4%; and in 0.4% of the measures, the bias reached ≥ 10%. Large bias was more frequent with the Nonin oximeter due to its systematic negative bias. These characteristics explain the clinical impact of the tested oximeters. Nonin oximeter was able to detect 100% of hypoxemia, whereas the Philips oximeter only detected 11%. In opposition, the oxygen weaning was much more frequent with the Philips oximeter in comparison with Nonin oximeter when usual SpO2 targets were utilized. The characteristics of the oximeters should be known by the users to adequately define the SpO2 target for patients based on the oximeter used. To our knowledge, this is the most recent study that compared several oximeters to concomitant SaO2 in ICU subjects.

The data presented in this study revealed that oximeters had a larger bias when tested critically ill subjects rather than healthy subjects. Previous studies conducted in healthy subjects found bias of −0.21% for Nonin, and −0.27% for Nellcor, on light-skinned healthy subjects (pigmentation similar to our population).26 Another study conducted on healthy subjects found bias of about +1.0 to −1.5% for Nonin oximeter, of about −1 to +1% for Masimo, and of about −0.5% to +1% for Nellcor on light-skinned subjects.27 In the present study, Nonin oximeter bias found in subjects was much greater than those found in previous studies conducted on healthy subjects, which might in part be related to the light-skin pigmentation (> 95% of the subjects included in the study). Pulse oximetry has been shown to overestimate SaO2 in dark-skin patients.20,36 Therefore, with the Nonin oximeter, the bias might be lower (less underestimation) on dark-skinned patients, whereas for the Philips the bias would likely be greater.36 Several other conditions that may affect arterial perfusion and oximeter accuracy were present in the subjects included in the present study including peripheral vascular disease (15% of included subjects), left ventricular dysfunction (11%), utilization of vasopressors (34%), as well as the presence of cardiovascular conditions such as diabetes (30%). In addition, studies that were carried out in healthy subjects were conducted under somewhat artificial conditions with subjects sitting still with each hand wrapped in a warming pad to ensure a good circulation to the fingers.26,27,31

It has been recently highlighted that the bias between SaO2 and SpO2 related to skin pigmentation is more pronounced in studies conducted in subjects compared with studies conducted in healthy volunteers.39 This is probably also true for the evaluation of oximeters, where the differences are greater in patients.

A limited number of studies comparing several oximeters have been recently conducted in subjects.21,22,38 In those studies, none have made a systematic comparison of 4 oximeters with simultaneous arterial blood gases. A multi-center study in New Zealand/Australia collected 400 pairs of SpO2 and SaO2, but only one oximeter was evaluated for each subject.22 Another study, conducted on mechanically ventilated subjects in the ICU in India, only compared 2 oximeters, Nonin and Philips.21 Results from this study slightly diverge from our results, probably as a consequence of the skin pigment of the subjects. In comparison to our data, the bias found in this study was of similar amplitude between oximeters (+2.49 ± 2.99 for Philips and 0.46 ± 1.68 for Nonin) but shifted by +2–3%. The impact of skin pigmentation on oximeters' bias has already been described.20,26,36 Oximeters potentially overestimate SaO2 in patients with dark-skin pigmentation, which can explain the difference in the bias found in Singh et al21 study and in our study. One recently published study evaluated oximeters' accuracy according to skin pigmentation but did not compare different oximeters and did not specify which oximeter was used.36 However, it highlighted greater overestimation of SaO2 in dark-skinned subjects. Moreover, a recent study has shown that skin pigmentation was associated with the risk of hidden hypoxemia and increased mortality.40

Oximeters characteristics, such as the tendency to underestimate or overestimate, may have a clinical impact, and this should be taken into account in clinical practice. In the present study, the Nonin oximeter underestimated SaO2 in 91.3% of the measurements. For clinicians, this means that the SaO2 of the patient is probably above the displayed SpO2. With other tested oximeters, it is more difficult for the clinician to conclude if the SaO2 is above or below the displayed SpO2.

The Nonin oximeter has the largest bias but in return is more sensitive for the detection of hypoxemia. However, an oximeter that underestimates SaO2 can lead to exposing patients to hyperoxemia, which can also be deleterious.5–12 Therefore, clinicians need to decrease SpO2 targets when using this type of oximeter to avoid hyperoxemia.

On the other hand, an oximeter that mainly overestimates SaO2, such as the Philips oximeter, lowers the risks of exposing patients to hyperoxemia. However, the main objective of oximeters (ie, to detect hypoxemia) may be missed. Therefore, clinicians may need to increase SpO2 targets when using this type of oximeter to avoid hypoxemia.

Concerning the oximeters that have low biases, as Nellcor and Masimo, SpO2 targets should not be adjusted, but clinicians must be aware of the weak accuracy of oximeters. This means that for a given SpO2 value displayed SaO2 may be as frequently both above or below expected.

There are potential clinical impacts that arise from a 2–3% bias between SaO2 and oximeters and from an even larger gap with several oximeters. Discrepancies related to skin pigmentation leading to occult hypoxemia in patients with dark skin have been described with potential impact on outcomes.36,40–42 In addition, the clinical decision to titrate oxygen may differ when using different SpO2 target recommendations13–15,43 and when using different oximeters. Our findings highlight the need for clinicians to be aware of the characteristics of the oximeter they use in their unit to avoid hypoxemia and hyperoxia and to adjust their SpO2 target. Current recommendations for SpO2 targets do not consider the variability related to oximeters used.13–15 However, the present study tends to show that the variability between the oximeters and the difference between the current SpO2 recommendations have a synergistic effect on the oxygenation parameters. Indeed, the O2 needs of a patient monitored with Philips oximeter following SpO2 target recommendations of 90–94%15 would be greatly underestimated when compared with the O2 needs of a patient monitored with Nonin oximeter following SpO2 target recommendations of 94–98%.13 It has been shown that small variations of SpO2 targets may have a major impact on oxygen requirements.16,18,19 A secondary analysis is currently underway to further analyze the impact of the oximeters' inaccuracy on O2 management and on multiple clinical scores that use SpO2. In patients with ARDS, there is a potential risk of occult hypoxemia with an oximeter that overestimates SaO2 (Philips). This was shown to be potentially deleterious in severe subjects with shock.44 On the contrary, there is a risk to use unnecessary high FIO2 with oximeters that underestimate SaO2 (Nonin), with potential risk of denitrogenation atelectasis and delayed oxygen weaning. If the SpO2 target is adjusted to the oximeter used, these risks may be reduced. Another clinical consequence is that SpO2 targets with automated oxygen titration are now decreased to 90% (and 88% in patients with COPD) in our institution as a Nonin oximeter is used with this device.16

Our findings suggest that targets used for the titration of O2 should be adapted to the oximeter used. Moreover, oxygen-free days has recently been proposed as a possible outcome marker in clinical trials,45 and this parameter may be influenced significantly by the type of oximeter used.46

Our study has a number of limitations. The number of subjects included was relatively low compared to large databases.40 However, our method assured that the SpO2 values and the SaO2 values were taken exactly concomitantly, in subjects who had been stable for several minutes, whereas in studies using large databases the SaO2 value was matched with the SpO2 value within 5–10 min.36,40–42 Knowing that oxygenation may vary within seconds after activity the only rigorous comparison of SaO2 and SpO2 requires concomitant measurements of SpO2 and SaO2 as we designed the study. Moreover, in the present study, the sampling made it possible to answer the main hypothesis: to demonstrate a difference of 0.5% between SaO2 and oximeters SpO2.

Our study's main limitation is the low diversity of skin pigmentation of our subjects. More than 96% of the subjects had a Fitzpatrick score of 1 or 2, which reflects the local population but limits the applicability of our results to this specific population. Similar comparisons are required on patients with darker-skin pigmentation given the significant potential impact.36,40 In terms of detection of hypoxemia, we had a limited number of subjects with SaO2 < 90%, and these results must be confirmed. The absence of perfusion index value can also be raised as a limitation. However, our subjects were stable, thus not in shock, and a member of the research team made sure each patient had good perfusion before the measurements were taken. The RMSE was higher for the Nonin oximeter in the present study. However, even if it is required by the FDA, RMSE is not the best measure of oximeter accuracy and is unintelligible for most bedside clinicians. It is likely that the oximeters' bias are shifted in patients with dark pigmentation. It was around +0.5 with Nonin and +2.5% with Philips oximeter, as shown in Singh study conducted in India, leading to higher RMSE for Philips. Consequently, the RMSE value is probably more representative when included subjects have balanced skin pigmentations, which is not the case here. It should be reported for each subjects' subgroups of skin pigmentation as recently proposed.47

Conclusions

The present study shows that bias and accuracy of oximeters are variable. Each oximeter behaves differently; some overestimate the oxygenation (Philips FAST); some underestimate the oxygenation (Nonin embedded in the FreeO2), and others have mean SpO2 values close to the mean SaO2 value (Nellcor N-600 and Masimo Radical-7) but with significant inaccuracy. With the latter, it is thus difficult for clinicians to determine whether these overestimate or underestimate oxygenation. All 4 of the tested oximeters showed a moderate correlation with SaO2. Therefore, oximeters should be used knowing their limits; and when SpO2 is used by clinicians to target the oxygen flow or FIO2, the oxygenation targets should probably be adapted based on the oximeter and skin pigmentation. For example, with guidelines that recommend a target SpO2 between 92–96%, on a population with the same characteristics as studied in this paper, this target might be increased to 93–97% with the Philips oximeter tested in the study and decreased to 89–93% with the Nonin oximeter tested in the study. More values could be required to assess these guidelines modification suggestions.

These marked differences may partly explain the major variations in recent recommendations for oxygen therapy: from 90–94%15 to 92–96%14 to 94–98%.13,43,48 Despite the fact that the first pulse oximeters were described 50 years ago, and as these devices are now used daily by clinicians around the world, their characteristics should be better known to optimize their utilization for titrating and monitoring oxygen therapy.

Footnotes

  • Correspondence: François Lellouche MD PhD, Centre de recherche de l'Institut universitaire de cardiologie et de pneumologie de Québec, 2725, Chemin Sainte-Foy, Québec, QC, Canada, G1V 4G5. E-mail: francois.lellouche{at}criucpq.ulaval.ca

  • See the Related Editorial on Page 708

  • Dr L'Her is a co-inventor of the FreeO2 device used in this study and a co-founder of OxyNov Inc. Dr L'Her discloses relationships with GE Healthcare, Sedana Medical, Vygon, and Archeon. Mr Branson discloses relationships with Lungpacer, Engineered Medical Systems, Ventec Life Systems, and Vyaire. Mr Branson is Editor-in-Chief of Respiratory Care. Dr Lellouche is a co-inventor of the FreeO2 device used in this study and a co-founder of OxyNov Inc. The remaining authors have disclosed no conflicts of interest.

  • Supplementary material related to this paper is available at http://www.rcjournal.com.

  • ClinicalTrials.gov registration NCT04772183.

REFERENCES

  1. 1.↵
    1. Galatius-Jensen S,
    2. Hansen J,
    3. Rasmussen V,
    4. Bildsoe J,
    5. Therboe M,
    6. Rosenberg J
    . Nocturnal hypoxemia after myocardial infarction: association with nocturnal myocardial ischemia and arrhythmias. Br Heart J 1994;72(1):23–30.
    OpenUrlAbstract/FREE Full Text
  2. 2.
    1. Gill NP,
    2. Wright B,
    3. Reilly CS
    . Relationship between hypoxemic and cardiac ischemic events in the perioperative period. Br J Anaesth 1992;68(5):471–473.
    OpenUrlCrossRefPubMed
  3. 3.
    1. Jubran A
    . Pulse oximetry. Crit Care 2015;19(1):272.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Sun Z,
    2. Sessler DI,
    3. Dalton JE,
    4. Devereaux PJ,
    5. Shahinyan A,
    6. Naylor AJ,
    7. et al
    . Postoperative hypoxemia is common and persistent: a prospective blinded observational study. Anesth Analg 2015;121(3):709–715.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Aubier M,
    2. Murciano D,
    3. Fournier M,
    4. Milic-Emili J,
    5. Pariente R,
    6. Derenne JP
    . Central respiratory drive in acute respiratory failure of patients with chronic obstructive pulmonary disease. Am Rev Respir Dis 1980;122(2):191–199.
    OpenUrlPubMed
  6. 6.
    1. Austin MA,
    2. Wills KE,
    3. Blizzard L,
    4. Walters EH,
    5. Wood-Baker R
    . Effect of high-flow oxygen on mortality in chronic obstructive pulmonary disease patients in prehospital setting: randomized controlled trial. BMJ 2010;341:c5462.
    OpenUrlAbstract/FREE Full Text
  7. 7.
    1. Cameron L,
    2. Pilcher J,
    3. Weatherall M,
    4. Beasley R,
    5. Perrin K
    . The risk of serious adverse outcomes associated with hypoxemia and hyperoxemia in acute exacerbations of COPD. Postgrad Med J 2012;88(1046):684–689.
    OpenUrlAbstract/FREE Full Text
  8. 8.
    1. Chu DK,
    2. Kim LH,
    3. Young PJ,
    4. Zamiri N,
    5. Almenawer SA,
    6. Jaeschke R,
    7. et al
    . Mortality and morbidity in acutely ill adults treated with liberal versus conservative oxygen therapy (IOTA): a systematic review and meta-analysis. Lancet 2018;391(10131):1693–1705.
    OpenUrlCrossRefPubMed
  9. 9.
    1. Farquhar H,
    2. Weatherall M,
    3. Wijesinghe M,
    4. Perrin K,
    5. Ranchord A,
    6. Simmonds M,
    7. et al
    . Systematic review of studies of the effect of hyperoxia on coronary blood flow. Am Heart J 2009;158(3):371–377.
    OpenUrlCrossRefPubMed
  10. 10.
    1. McNulty PH,
    2. King N,
    3. Scott S,
    4. Hartman G,
    5. McCann J,
    6. Kozak M,
    7. et al
    . Effects of supplemental oxygen administration on coronary blood flow in patients undergoing cardiac catheterization. Am J Physiol Heart Circ Physiol 2005;288(3):H1057–1062.
    OpenUrlCrossRefPubMed
  11. 11.
    1. Floyd TF,
    2. Ratcliffe SJ,
    3. Detre JA,
    4. Woo YJ,
    5. Acker MA,
    6. Bavaria JE,
    7. et al
    . Integrity of the cerebral blood-flow response to hyperoxia after cardiopulmonary bypass. J Cardiothorac Vasc Anesth 2007;21(2):212–217.
    OpenUrlPubMed
  12. 12.↵
    1. Stub D,
    2. Smith K,
    3. Bernard S,
    4. Nehme Z,
    5. Stephenson M,
    6. Bray JE,
    7. et al
    ; AVOID Investigators. Air versus oxygen in ST-segment elevation myocardial infarction. Circulation 2015;131(24):2143–2150.
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    1. O'Driscoll BR,
    2. Howard LS,
    3. Earis J,
    4. Mak V
    ; British Thoracic Society Emergency Oxygen Guideline Group; Group BTSEOGD Group. BTS guideline for oxygen use in adults in health care and emergency settings. Thorax 2017;72(Suppl 1):ii1–ii90.
    OpenUrlFREE Full Text
  14. 14.↵
    1. Beasley R,
    2. Chien J,
    3. Douglas J,
    4. Eastlake L,
    5. Farah C,
    6. King G,
    7. et al
    . Thoracic Society of Australia and New Zealand oxygen guidelines for acute oxygen use in adults: “Swimming between the flags.” Respirology 2015;20(8):1182–1191.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Siemieniuk RAC,
    2. Chu DK,
    3. Kim LH,
    4. Guell-Rous MR,
    5. Alhazzani W,
    6. Soccal PM,
    7. et al
    . Oxygen therapy for acutely ill medical patients: a clinical practice guideline. BMJ 2018;363:k4169.
    OpenUrlFREE Full Text
  16. 16.↵
    1. Lellouche F,
    2. L'Her E
    . Automated oxygen flow titration to maintain constant oxygenation. Respir Care 2012;57(8):1254–1262.
    OpenUrlAbstract/FREE Full Text
  17. 17.
    1. Lellouche F,
    2. Bouchard PA,
    3. Roberge M,
    4. Simard S,
    5. L'Her E,
    6. Maltais F,
    7. et al
    . Automated oxygen titration and weaning with FreeO2 in patients with acute exacerbation of COPD: a pilot randomized trial. Int J Chron Obstruct Pulmon Dis 2016;11:1983–1990.
    OpenUrlPubMed
  18. 18.↵
    1. L'Her E,
    2. Dias P,
    3. Gouillou M,
    4. Riou A,
    5. Souquiere L,
    6. Paleiron N,
    7. et al
    . Automatic versus manual oxygen administration in the emergency department. Eur Respir J 2017;50(1).
  19. 19.↵
    1. L'Her E,
    2. Jaber S,
    3. Verzilli D,
    4. Jacob C,
    5. Huiban B,
    6. Futier E,
    7. et al
    . Automated closed-loop versus standard manual oxygen administration after major abdominal or thoracic surgery: an international multi-center randomized controlled study. Eur Respir J 2021;57(1).
  20. 20.↵
    1. Jubran A,
    2. Tobin MJ
    . Reliability of pulse oximetry in titrating supplemental oxygen therapy in ventilator-dependent patients. Chest 1990;97(6):1420–1425.
    OpenUrlCrossRefPubMed
  21. 21.↵
    1. Singh AK,
    2. Sahi MS,
    3. Mahawar B,
    4. Rajpurohit S
    . Comparative evaluation of accuracy of pulse oximeters and factors affecting their performance in a tertiary intensive care unit. J Clin Diagn Res 2017;11(6):OC05–OC08.
    OpenUrl
  22. 22.↵
    1. Pilcher J,
    2. Ploen L,
    3. McKinstry S,
    4. Bardsley G,
    5. Chien J,
    6. Howard L,
    7. et al
    . A multi-center prospective observational study comparing arterial blood gas values to those obtained by pulse oximeters used in adult patients attending Australian and New Zealand hospitals. BMC Pulm Med 2020;20(1):7.
    OpenUrl
  23. 23.
    1. Van de Louw A,
    2. Cracco C,
    3. Cerf C,
    4. Harf A,
    5. Duvaldestin P,
    6. Lemaire F,
    7. et al
    . Accuracy of pulse oximetry in the intensive care unit. Intensive Care Med 2001;27(10):1606–1613.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Clayton DG,
    2. Webb RK,
    3. Ralston AC,
    4. Duthie D,
    5. Runciman WB
    . A comparison of the performance of 20 pulse oximeters under conditions of poor perfusion. Anaesthesia 1991;46(1):3–10.
    OpenUrlPubMed
  25. 25.↵
    ISO. 80601–2- 61:2011, Medical electrical equipment — Part 2–61: Particular requirements for basic safety and essential performance of pulse oximeter equipment.
  26. 26.↵
    1. Bickler PE,
    2. Feiner JR,
    3. Severinghaus JW
    . Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology 2005;102(4):715–719.
    OpenUrlCrossRefPubMed
  27. 27.↵
    1. Feiner JR,
    2. Severinghaus JW,
    3. Bickler PE
    . Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender. Anesth Analg 2007;105(6 Suppl):S18–s23.
    OpenUrlCrossRefPubMed
  28. 28.
    1. Henry NR,
    2. Hanson AC,
    3. Schulte PJ,
    4. Warner NS,
    5. Manento MN,
    6. Weister TJ,
    7. Warner MA
    . Disparities in hypoxemia detection by pulse oximetry across self-identified racial groups and associations with clinical outcomes. Crit Care Med 2022;50(2):205–211.
    OpenUrl
  29. 29.
    1. Crooks CJ,
    2. West J,
    3. Morling JR,
    4. Simmonds M,
    5. Juurlink I,
    6. Briggs S,
    7. et al
    . Pulse oximeter measurements vary across ethnic groups: an observational study in patients with COVID-19. Eur Respir J 2022;59(4).
  30. 30.↵
    1. Tobin MJ,
    2. Jubran A
    . Pulse oximetry, racial bias, and statistical bias. Ann Intensive Care 2022;12(1):2.
    OpenUrl
  31. 31.↵
    1. Louie A,
    2. Feiner JR,
    3. Bickler PE,
    4. Rhodes L,
    5. Bernstein M,
    6. Lucero J
    . Four types of pulse oximeters accurately detect hypoxia during low perfusion and motion. Anesthesiology 2018;128(3):520–530.
    OpenUrlCrossRefPubMed
  32. 32.↵
    1. Basaranoglu G,
    2. Bakan M,
    3. Umutoglu T,
    4. Zengin SU,
    5. Idin K,
    6. Salihoglu Z
    . Comparison of SpO2 values from different fingers of the hands. Springerplus 2015;4:561.
    OpenUrl
  33. 33.↵
    1. Hinkelbein J,
    2. Koehler H,
    3. Genzwuerker HV,
    4. Fiedler F
    . Artificial acrylic finger nails may alter pulse oximetry measurement. Resuscitation 2007;74(1):75–82.
    OpenUrlCrossRefPubMed
  34. 34.↵
    1. Nesseler N,
    2. Frénel JV,
    3. Launey Y,
    4. Morcet J,
    5. Mallédant Y,
    6. Seguin P
    . Pulse oximetry and high-dose vasopressors: a comparison between forehead reflectance and finger transmission sensors. Intensive Care Med 2012;38(10):1718–1722.
    OpenUrlCrossRefPubMed
  35. 35.↵
    1. Wilson BJ,
    2. Cowan HJ,
    3. Lord JA,
    4. Zuege DJ,
    5. Zygun DA
    . The accuracy of pulse oximetry in emergency department patients with severe sepsis and septic shock: a retrospective cohort study. BMC Emerg Med 2010;10:9.
    OpenUrlCrossRefPubMed
  36. 36.↵
    1. Sjoding MW,
    2. Dickson RP,
    3. Iwashyna TJ,
    4. Gay SE,
    5. Valley TS
    . Racial bias in pulse oximetry measurement. N Engl J Med 2020;383(25):2477–2478.
    OpenUrlCrossRefPubMed
  37. 37.↵
    1. Fitzpatrick TB
    . The validity and practicality of sun-reactive skin types I through VI. Arch Dermatol 1988;124(6):869–871.
    OpenUrlCrossRefPubMed
  38. 38.↵
    1. Pu LJ,
    2. Shen Y,
    3. Lu L,
    4. Zhang RY,
    5. Zhang Q,
    6. Shen WF
    . Increased blood glycohemoglobin A1c levels lead to overestimation of arterial oxygen saturation by pulse oximetry in patients with type 2 diabetes. Cardiovasc Diabetol 2012;11:110.
    OpenUrlCrossRefPubMed
  39. 39.↵
    1. Okunlola O,
    2. Lipnick MS,
    3. Batchelder P,
    4. Bernstein M,
    5. Feiner J,
    6. Bickler P
    . Pulse oximeter performance, racial inequity, and the work ahead. Respir Care 2022;67(2):252–257.
    OpenUrlAbstract/FREE Full Text
  40. 40.↵
    1. Wong AI,
    2. Charpignon M,
    3. Kim H,
    4. Josef C,
    5. de Hond AAH,
    6. Fojas JJ,
    7. et al
    . Analysis of discrepancies between pulse oximetry and arterial oxygen saturation measurements by race and ethnicity and association with organ dysfunction and mortality. JAMA Netw Open 2021;4(11):e2131674.
    OpenUrl
  41. 41.
    1. Valbuena VSM,
    2. Seelye S,
    3. Sjoding MW,
    4. Valley TS,
    5. Dickson RP,
    6. Gay SE,
    7. et al
    . Racial bias and reproducibility in pulse oximetry among medical and surgical inpatients in general care in the Veterans Health Administration 2013–19: multi-center, retrospective cohort study. BMJ 2022;378:e069775).
    OpenUrlAbstract/FREE Full Text
  42. 42.↵
    1. Fawzy A,
    2. Wu TD,
    3. Wang K,
    4. Robinson ML,
    5. Farha J,
    6. Bradke A,
    7. et al
    . Racial and ethnic discrepancy in pulse oximetry and delayed identification of treatment eligibility among patients with COVID-19. JAMA Intern Med 2022;182(7):730–738.
    OpenUrl
  43. 43.↵
    1. Piraino T,
    2. Madden M,
    3. K JR,
    4. Lamberti J,
    5. Ginier E,
    6. S LS
    . Management of adult patients with oxygen in the acute care setting. Respir Care 2022;67(1):115–128.
    OpenUrlAbstract/FREE Full Text
  44. 44.↵
    1. Barrot L,
    2. Asfar P,
    3. Mauny F,
    4. Winiszewski H,
    5. Montini F,
    6. Badie J,
    7. et al
    ; LOCO2 Investigators and REVA Research Network. Liberal or conservative oxygen therapy for acute respiratory distress syndrome. N Engl J Med 2020;382(11):999–1008.
    OpenUrlCrossRefPubMed
  45. 45.↵
    1. Moskowitz A,
    2. Shotwell MS,
    3. Gibbs KW,
    4. Harkins M,
    5. Rosenberg Y,
    6. Troendle J,
    7. et al
    . Oxygen-free days as an outcome measure in clinical trials of therapies for COVID-19 and other causes of new-onset hypoxemia. Chest 2022;162(4):804–814.
    OpenUrl
  46. 46.↵
    1. Lellouche F,
    2. Blanchet M,
    3. Branson R
    . Oxygen-free days and the confounders of clinical practice. Chest Dec 2022;162(6):e331–e332.
    OpenUrl
  47. 47.↵
    1. Sjoding MW,
    2. Iwashyna TJ,
    3. Valley TS
    . Change the framework for pulse oximeter regulation to ensure clinicians can give patients the oxygen they need. Am J Respir Crit Care Med 2023;207(6):661–664.
    OpenUrl
  48. 48.↵
    1. O'Driscoll BR,
    2. Howard LS,
    3. Earis J,
    4. Mak V
    . British Thoracic Society Guideline for oxygen use in adults in health care and emergency settings. BMJ Open Respir Res 2017;4(1):e000170.
    OpenUrlFREE Full Text
Back to top

In this issue

 

Print
Download PDF
Article Alerts
Email Article
Citation Tools

Jump to section

Related Articles

Cited By...

Keywords