## Abstract

**BACKGROUND:** Blood gas quality control (QC) is an essential and mandatory part of a laboratory’s quality plan. The acceptable QC range should be 2 SD from the mean value. The use of assayed QC material does not negate the responsibility of the laboratory to calculate the mean and 2 SD ranges of QC measurements for verification. Verifying assayed QC ranges is a Clinical Laboratory Improvement Amendment (CLIA) requirement. This study shows the results of assayed QC mean and 2 SD range verification from a blood gas analyzer.

**METHODS:** QC data from a blood gas analyzer were compared to manufacturer-provided mean and ranges. The percent difference between the measured mean and the manufacturer-provided mean was calculated to assess agreement. The measured SD was used to determine how many SD the manufacturer-provided ranges were from the measured mean.

**RESULTS:** The largest difference in mean values was 2.27% > the manufacturer-provided mean. Forty-eight percent of all mean value comparisons showed a difference of 0%, and 71% were < 1%. The manufacturer-provided ranges were considerably wider than the measured 2 SD range, ranging from 2.4–75 SD. None of the manufacturer-provided ranges were deemed acceptable for clinical use.

**CONCLUSIONS:** Our analysis validates the CLIA mandate and American Association for Respiratory Care Clinical Practice Guideline recommendation that laboratories must verify manufacturer-provided QC means and ranges and adjust QC means and ranges to match the performance of their blood gas analyzer.

## Introduction

Quality control (QC) is an essential and mandatory part of a laboratory’s quality plan. However, there is significant variability in QC practices even among some of the most prestigious hospitals.^{1} Many laboratories that perform blood gas analysis use assayed materials to perform QC testing. Assayed QC materials have manufacturer-provided ranges, whereas unassayed QC materials require the user to calculate the mean and ranges from a series of measurements. However, the use of assayed QC materials does not negate the responsibility of the laboratory to calculate the mean and ranges of QC measurements. The laboratory must make these calculations to verify that the manufacturer-provided QC mean and ranges are appropriate for their analyzer. Verifying assayed QC ranges is a Clinical Laboratory Improvement Amendment (CLIA) requirement (§493.1256 standard: control procedures [d][10][ii]): “the laboratory may use the stated value of a commercially assayed control material provided the stated value is for the methodology and instrumentation employed by the laboratory and is verified by the laboratory” (https://ecfr.federalregister.gov/current/title-42/chapter-IV/subchapter-G/part-493/subpart-K/subject-group-ECFRc96daead380f6ed/section-493.1256#p-493.1256(d)(10)(ii). *Accessed August 2, 2021*). In addition, the American Association for Respiratory Care (AARC) Clinical Practice Guideline for blood gas analysis and hemoximetry states that “the laboratory director or designee should determine the acceptable range for QC results, based on statistically relevant or medical-needs criteria.”^{2}

If the manufacturer-provided QC mean values and ranges are not representative of the true performance of the analyzer, QC testing may not alert the laboratory’s clinicians that a problem exists. Such a situation may affect the accuracy of patient data and potentially affect patient care. Unfortunately, some laboratories that perform blood gases analysis use manufacturer-provided QC mean values and ranges without performing verification. Whereas there is a paucity of data regarding the frequency of blood gas laboratories using unverified QC ranges, based on informal conversations we are confident that this is not an uncommon practice. This study shows the results of assayed QC mean and range verification from a blood gas analyzer and underscores the importance of verifying and adjusting QC mean values and ranges to match the performance of the analyzer.

### QUICK LOOK

#### Current Knowledge

Some laboratories that perform blood gases analysis use assayed materials to perform quality control (QC) testing. Assayed QC materials have manufacturer-provided mean values and ranges, whereas unassayed QC materials require the user to calculate the mean values and ranges from a series of measurements. Clinical Laboratory Improvement Amendment (CLIA) and the American Association for Respiratory Care (AARC) recommend that blood gas laboratories verify that manufacturer-provided QC mean values and ranges are appropriate for their blood gas analyzer.

#### What This Paper Contributes to Our Knowledge

Our analysis validates the CLIA mandate and AARC Clinical Practice Guideline recommendation that laboratories must verify manufacturer-provided QC means and ranges and adjust QC means and ranges to match the performance of their blood gas analyzer. Verification of assayed QC mean values and ranges is necessary to ensure quality testing and patient safety.

## Methods

QC data from a blood gas analyzer (Roche cobas b 221, Roche Diagnostics, Mannheim, Germany) were compared to manufacturer-provided mean values and ranges. Data points identified as random errors according to Westgard’s rules (http://www.westgard.com. *Accessed August 2, 2021*) were excluded from analysis. The manufacturer-provided QC material (AUTO-TROL PLUS B, Roche Diagnostics) lot numbers were as follows: level 1, 21480768; level 2, 21480863; level 3, 21480962. Measured QC mean values and SD were calculated by the Roche cobas b 221 software. Figures 1 and 3 were generated using statistical software (Prism 4, GraphPad Software, San Diego, California). Figure 2 was generated using Microsoft Excel (Microsoft, Redmond, Washington).

The percent difference between the measured mean and the manufacturer-provided mean was calculated to assess agreement. The measured SD was used to determine how many SD the manufacturer-provided ranges were from the measured mean.

## Results

The agreement between the measured and manufacturer-provided QC mean values and ranges are listed in Table 1. The largest difference in mean values was observed in the level 2 P_{CO2}, where the measured mean was 2.27% (1 mm Hg) > the manufacturer-provided mean. Forty-eight percent of all mean value comparisons showed a difference of 0%, and 71% were < 1%. The manufacturer-provided ranges were considerably wider than the measured 2 SD range, ranging from 2.4 SD (low range of level 1 P_{CO2}) to 75 SD (levels 2 and 3 methemoglobin). An example of the assayed ranges in comparison to the measured 2 SD range is shown in a Levey-Jennings plot for level 1 P_{O2} in Figure 1, where the low range is 8.7 and the high range is 10.2 SD from the measured mean. None of the manufacturer-provided ranges was deemed acceptable for clinical use.

## Discussion

An analysis of an electronic error reporting system from 30 health care organizations showed that laboratory errors accounted for 14.1% of all quality events.^{3} Laboratory errors may occur during the pre-analytical, analytical, and post-analytical phases. Whereas pre-analytical errors account for most laboratory events,^{3} a malfunctioning analyzer is an obvious source of error and is the primary focus of a QC program.

QC is an essential and mandatory part of a laboratory’s quality plan. The acceptable QC range should be 2 SD from the mean value. As mentioned earlier, manufacturers may provide “assayed” expected mean values and ranges, but these must be verified as accurate reflections of the analyzer’s performance. To verify assayed QC mean values and ranges, a minimum of 20 QC measurements (same level from the same lot) should be collected from multiple successive days. Any obvious and infrequent outliers or “random errors” (eg, expected P_{O2} value ∼ 50 mm Hg, measured value 100 mm Hg) should be excluded from analysis. If random errors are occurring frequently, the analyzer should be inspected for malfunctions. Using < 20 samples to calculate the mean and SD may not provide an accurate assessment of the analyzer’s performance, and using a very large sample (eg, > 100 values) unnecessarily adds time to the process. Computer software can quickly and accurately calculate mean and SD values; however, laboratory personnel should still understand how these values are derived and what they represent.

The mean or average (denoted by the symbol or μ) is derived by adding the sum of all recorded values and dividing that value by the number of measurements as shown in the equation below:

Where

= mean

Σ^{x} = sum of recorded values

n = number of values.

SD is a measure of the variability around the mean. Two blood gas analyzers could produce identical mean values for a QC measure (eg, level 1 P_{O2}); however, the analyzer with the smaller SD has less variability and more precision. Less variability and more precision impact the confidence one should have in patient data because repeating analysis on a blood specimen will not yield exactly the same results. It stands to reason that the data variability present in QC testing may also apply to results reported from patient sampling. For example, consider two identical blood gas analyzers, both report similar mean values for level 1 P_{O2}, 60 mm Hg; however, the observed range of measured values for analyzer #1 is 55–65 mm Hg, and the observed range of measured values is 45–75 mm Hg for analyzer #2. Clinicians should have less confidence that a patient’s reported P_{O2} of 65 mm Hg is truly > 60 mm Hg if analyzer #2 was used as opposed to analyzer #1. The precision of a blood gas analyzer can impact clinical decision-making especially when small differences in blood gas values change clinical classifications such as qualifying for long-term O_{2} therapy^{4} or determining ARDS severity.^{5}

The lower-case Greek letter σ is used to denote SD. SD calculation starts by summing the difference between each measured value and the mean. For example, if the QC mean is 60, a value of 55 has a difference score of −5, and a value of 65 has a difference score of +5. Because the mean is by definition at the center of all values, the sum of all difference scores will always be zero. To calculate SD, the difference scores will need to be summed; and because some are negative and some are positive, they are squared to transform all difference scores to positive numbers. Now the SD can be calculated by the following equation: where

σ = SD

Σ = sum

x_{i} = individual values

= mean of all values

n = number of values.

The SD equation subtracts 1 from the number of measurements (one degree of freedom) to calculate a slightly larger SD. Using a larger SD makes the sample variance more generalizable to the entire population. For example, if you were calculating the mean and SD of resting S_{pO2} from a sample of 1,000 subjects in a city with a population 100,000, using the number of samples without subtracting 1 might indicate less variance around the mean (ie, smaller SD) than if you sampled all 100,000 residents. This process (n-1) is intended to produce a less biased estimate of variance. Because the SD of blood gas QC is calculated from a sample (eg, 20 QC measurements) and not the total number of measurements projected for the QC lot (eg, 365 measurements for a lot that expires in 1 year), a slightly higher SD is desirable to avoid underestimating the true variance that might be higher if calculated from all 365 measurements in a 1-y period. Using a SD smaller than the true variance for blood gas QC will result in more false “out of control” conditions.

A 2 SD range is used for blood gas QC because the empirical rule states that if the blood gas analyzer is operating normally, and the QC material is unadulterated, 95% of recorded QC values should be within 2 SD of the mean.^{6} Therefore, a QC value 2 SD from the mean should only occur once in 20 measurements (5%). If a recorded QC measurement is between 2–3 SD from the mean, a “warning” condition exists because this is an unusual value (1/20) based on past performance. The empirical rule also states that 99.7% of QC results should be within 3 SD of the mean, so a QC value > 3 SD from the mean should never occur if the analyzer is operating normally and the QC material is unadulterated. Figure 2 shows the distribution or “bell” curve for level 1 P_{O2} data. The mean value is at the center of the distribution curve; 68%, 95%, and 99.7% of QC measurements should fall between 1 SD, 2 SD, and 3 SD, respectively. A Levey-Jennings plot charts QC data against the mean and SD as a function of time; whereas the distribution curve is not displayed, it is essentially flipped 90°. Figure 3 shows a cartoon drawing of the differences in the size of the distribution curves for measured versus assayed ranges on a Levey-Jennings plot for level 1 P_{O2}.

The software of modern blood gas analyzers will alert the laboratory personnel if a QC measurement is out of range, and many devices will lock out the analyzer from performing clinical testing until the QC passes. It is, therefore, important that QC ranges are an accurate reflection of the blood gas analyzer’s true performance.

Our analysis shows the importance of verifying manufacturer-provided QC ranges. Whereas the manufacturer-provided means were essentially identical to our measured mean values, with the exception of level 1 low P_{CO2} and level 2 high P_{CO2}, the manufacturer-provided ranges were markedly outside of 2 SD. This level of discordance could result in undetected analyzer malfunction and potentially impact clinical decision-making.

## Conclusions

Our analysis validates the CLIA mandate and AARC Clinical Practice Guideline recommendation^{2} that laboratories must verify manufacturer-provided QC mean values and ranges and adjust the QC mean value and ranges to match the performance of their blood gas analyzer.

## Footnotes

- Correspondence: Jeffrey M Haynes RRT RPFT FAARC, Pulmonary Function Laboratory, Elliot Health System, 185 Queen City Avenue, Manchester, New Hampshire 03101. E-mail: jhaynes{at}elliot-hs.org
Mr Haynes discloses a relationship with Morgan Scientific. Mr Fishwick has disclosed no conflicts of interest.

- Copyright © 2022 by Daedalus Enterprises