Abstract
BACKGROUND: The guidelines to conduct and interpret conventional pulmonary function (PFT) tests are frequently reviewed and updated. However, the quality assurance and quality control (QA/QC) guidelines for respiratory oscillometry testing remain limited. QA/QC guidelines are essential for oscillometry to be used as a diagnostic pulmonary function test (PFT) in a clinical setting.
METHODS: We developed a QA/QC protocol shortly after oscillometry was introduced in our laboratory as part of a clinical study. The first clinical study began after the research personnel completed 3 h of combined didactic and hands-on training and establishment of a standard operating protocol (SOP) for oscillometry testing. All oscillometry tests were conducted using the initial SOP protocol from October 17, 2017, to April 6, 2018. At this time, the first QA/QC audit took place, followed by revisions to the SOP, the addition of a QA/QC checklist, and the development of a 12-h training program. A second audit of oscillometry tests was conducted from April 9, 2018, to June 30, 2019. Both audits were completed by a registered cardiopulmonary technologist from the Toronto General Pulmonary Function Lab.
RESULTS: The first audit evaluated 197 paired oscillometry-PFT tests and found 10 tests (5.08%) to be invalid, with a coefficient of variation > 15%. The second audit examined 1,930 paired oscillometry-PFT tests; only 3 tests (0.16%) were unacceptable, with a coefficient of variation > 15%. Improvement in QA/QC was significantly better compared to the first audit (P < .001).
CONCLUSIONS: Although oscillometry requires minimal subject cooperation, application of the principles that govern the conduct and application of a PFT are important for ensuring that oscillometry testing is performed according to acceptability and reproducibility. Specifically, the inclusion of a SOP, a proper training program, a QA/QC checklist, and regular audits with feedback are vital to ensure that oscillometry is conducted accurately and precisely.
Introduction
Conventional pulmonary function tests (PFT), including spirometry, lung volumes, airway resistance, and diffusing capacity, are routinely ordered by physicians to aid in the diagnosis and management of lung diseases. PFT has a long-established history of use and was developed in the early 1800s by Davy, who used a hydrogen-dilution technique to determine lung volume.1 The invention of the spirometer in 1846 by Hutchinson,1 the diffusing capacity test in 1911 by Marie Krogh (cited in Graham et al),2 the timed volumes (including FEV1) by Tiffineau and Pinelli in 1947,3 and the airway resistance by DuBois et al4 in 1956 shaped the development of PFT in the modern world. Reference values, guidelines to conduct and interpret PFTs, and quality assurance measures for pulmonary function laboratories are regularly updated and reviewed.2,5-11 Pulmonary function laboratory accreditation by health regulatory bodies or health professional associations is available in some countries, including Canada, Australia, and New Zealand, where certification underscores the importance of qualified personnel, appropriate standard operating procedures (SOPs), robust quality control, and reliable and accurate lung function testing equipment.12 PFTs are considered the accepted standard, but they are known to be poorly sensitive for the evaluation of peripheral airways and tissues.
Respiratory oscillometry is an emerging PFT tool that measures respiratory mechanics and is particularly sensitive to small airways disease, a region of the lung not well evaluated by PFTs. Oscillometry is commonly known as a forced oscillation technique and uses pseudorandom noise to generate a multi-frequency signal or uses simply a mono-frequency signal; impulse oscillometry system is a subclass of forced oscillation technique that utilizes a train of impulses to generate the oscillation signals.13-19 Oscillometry measures respiratory system impedance, which describes the pressure-flow relationship at each frequency. The respiratory system impedance is composed of the respiratory resistance and the respiratory reactance.18 Respiratory resistance considers the pressure component in phase with flow, representing resistive losses, whereas respiratory reactance reflects impedance to changes in volume and volume acceleration (pressure components out of phase with flow) characterizing the elastance and inertance, respectively, of the respiratory system. Respiratory resistance increases with obstruction and becomes frequency-dependent with small airway obstruction or a proximal shunt effect.19-22 Respiratory reactance is particularly useful at low frequencies, where it is dominated by the stiffness of the lungs and the chest wall. Respiratory reactance becomes increasingly negative with both obstructive and restrictive disease and is sensitive to peripheral inhomogeneity.18-22 Unlike PFT, oscillometry requires minimal cooperation from patients because it is conducted during tidal breathing for 20 s. Hence, patients who have difficulty performing a forced maneuver, such as the elderly, the very young, or those with neuromuscular disease or a contraindication such as recent eye surgery, pulmonary embolism, or advanced pregnancy, could still be evaluated with oscillometry to assess PFT.
Oscillometry was first developed by DuBois et al18 in 1956 and was used extensively in the experimental setting. Guidelines for the implementation and use of oscillometry in clinical practice were published by the European Respiratory Society (ERS) in 2003.17 An American Thoracic Society/European Respiratory Society (ATS/ERS) Task Force reviewed oscillometry as a tool for PFT in preschool children in 2006.23
Oscillometry was introduced to the Toronto General Pulmonary Function Laboratory in October 2017 as part of the first several clinical research studies that compared oscillometry with PFT in identifying specific clinical outcomes. Our pulmonary function laboratory continues to operate on the principles that were developed by Woolf24 in 1968, which state that the operator conducting the test must be well-trained, reliable, and capable of working under minimal supervision. Our laboratory performs > 13,000 PFTs per year. To ensure quality control of testing, routine preventive maintenance and calibration of PFT equipment are performed regularly.2,5-11 Shortly after the launch of the first oscillometry research study, we recognized the need to develop a quality assurance (QA) and quality improvement program for oscillometry to ensure accurate and precise results. We hypothesized that the development and introduction of a QA and quality improvement program for oscillometry would ensure that acceptable and reproducible tests are obtained.
QUICK LOOK
Current knowledge
Quality assurance and quality control in a pulmonary function laboratory are important to ensure that tests are performed to ATS/ERS acceptability and reproducibility criteria.
What this paper contributes to our knowledge
Our audits revealed that investments in the training of personnel, establishment of a clear standard operating procedure, and the implementation of a QC checklist play crucial roles in ensuring the QA/QC of oscillometry. Our findings confirm that factors influencing acceptability and reproducibility of spirometry are small if the technician is well trained and experienced.
Methods
Development of Standard Operating Procedures for Oscillometry
The oscillometry clinical research studies were approved by the University Health Network Research Ethic Board and conducted in accordance with institutional guidelines. The study was conducted at the Pulmonary Function Laboratory of Toronto General Hospital, University Health Network, in Toronto, Ontario, Canada. Oscillometry was performed using the TremoFlo C-100 Airwave Oscillometry System (Thorasys Thoracic Medical Devices, Montréal, Canada). A simple standard operating procedure (SOP) following the ERS guidelines and manufacturer’s manual was developed for the first clinical study in October 2017. In brief, oscillometry was conducted with the subject wearing a nose clip, sitting upright with palms supporting the cheeks, and the thenar eminence supporting the floor of the mouth, the head in a neutral position, the tongue maintained forward and the legs uncrossed. Artifacts related to cough or glottis closure were automatically rejected and removed by the device. Only tests that achieved a coefficient of variation (mean ± SD) ≤ 15% were accepted; the coefficient of variation is a measure of reproducibility of repeated measurements.
Coherence is an indication of how close the oscillation flow and pressure are related at a given frequency and is mostly a reflection of the signal-to-noise ratio within each test. Although most oscillometry studies have reported coherence as a metric of quality control, the actual scheme of computation is not known and coherence acceptance limits are arbitrary. Instead, calculation of the coefficient of variation is preferred to quantitate variation between tests.17,19 A minimum of three 20-s recordings were conducted, with rests of 30 s between recordings. Biometric information including age, sex, height, and weight were also recorded at the time of oscillometry. PFTs were performed after conducting oscillometry. When PFTs were performed prior to conducting oscillometry, although this is strongly discouraged and occurred rarely (ie, 5 of 2,127 total paired oscillometry-PFTs), a minimum of 10 min rest was given to subjects to stabilize lung volume before first measurement of oscillometry.17 Lung volumes by body plethysmography is the last test conducted during PFTs, so the interaction between a maximum forced expiration maneuver affecting the stable bronchial tone was minimal.
Oscillometry was performed primarily by 2 research personnel and occasionally by registered cardiopulmonary technologists employed at the Toronto General Pulmonary Function Laboratory. The personnel were trained by Thorasys in 2 separate sessions, initially via teleconferencing and then in person, followed by practice sessions with each other. In total, 3 h of training occurred prior to testing study subjects.
Development of Quality Assurance and Control for the Conduct of Oscillometry
We adapted the QA and quality control (QA/QC) procedures for PFT that were in place in our laboratory for oscillometry in addition to the suggestions from the ERS guidelines and the Tremoflo user manual. As suggested by the manufacturer, the Tremoflo device was calibrated daily with the proprietary standard (calibration reference load, REF#100986, Thorasys Thoracic Medical Device, Montréal, Canada) prior to subject testing. Both the ERS guidelines17 and the Tremoflo user manual suggest that measurements should have a minimum duration of 6 s, valid data points ≥ 70%, and a minimum of 3 valid measurements with a coefficient of variation ≤ 15%. However, these suggested guidelines were not automatically excluded and must be assessed by the operator.
In our QA/QC procedures, we provided subjects with 3 min of rest prior to the first oscillometry measurement to stabilize lung volumes because it is a 60-m walk from the waiting area to the laboratory. A minimum of 3 tidal breaths must be observed prior to each oscillometry recording to ensure subjects are breathing at resting functional residual capacity level to avoid mechanical drifts. A 30 s rest is also given to subjects between each measurement to avoid short-term variability. We increased the minimum rest time from 3 min to 10 min to stabilize the lung volumes if forced respiratory maneuvers were performed prior to oscillometry test. Biological calibration was also added to ensure that the 2 oscillometry devices were functioning properly.
Biological calibration, also known as biological quality control (BioQC) was performed twice weekly with healthy personnel in the pulmonary function laboratory with the initial collection of a minimum of 10 tests on different days to establish the BioQC range. After that, biological calibration was performed monthly. BioQC of oscillometry is determined by evaluating the resistance at 5 Hz, the reactance at 5 Hz, and the reactance area. The means and standard deviations for these parameters are calculated. At least 2 BioQC subjects undergo oscillometry tests on the same day to avoid intra-individual variation (eg, due to illness). A BioQC measurement that exceeds the mean ± 2 SD is cautioned.9 In this scenario, when the BioQC measurement exceeds ± 2 SD, measurements are repeated a week later to monitor the mean value. If an observation exceeds the mean ± 3 SD, an “out of control” condition is identified.11 An “out of control” condition is also deemed to exist under the following situations: (1) 2 consecutive measurements that exceed mean ± 2 SD, (2) 4 consecutive measurements that exceed the mean ± 1 SD, or (3) 10 consecutive observations that fall on the same side of the mean.11 Whenever an “out of control” condition is observed, the device is not used for subject testing until it is verified to be in good working condition.11 PFTs are also performed on the same day to help assess daily variation between and within individuals. QA/QC of PFT is performed as part of the clinical operating standard of our laboratory.
Evaluation of Quality Assurance and Control
The first QA/QC audit was conducted by a registered cardiopulmonary technologist 5 months after introduction of oscillometry to the lab when 197 paired oscillometry-PFT tests were completed. Revisions to the SOP and an official oscillometry training program were developed at this time. An informal QA/QC audit was repeated 3 months later, and a formal audit was done at the end of June 2019. Differences between the 2 audits were compared using the chi-square test (Prism 6.0, GraphPad Software, La Jolla, California).
Results
From October 17, 2017, to April 6, 2018, a total of 197 paired oscillometry-PFTs studies were collected. Oscillometry was conducted primarily by 2 research personnel and occasionally by registered cardiopulmonary technologists. All PFTs were performed as part of the subject’s routine clinical care and were conducted by registered cardiopulmonary technologists according to ATS/ERS guidelines. Therefore, PFTs were not part of our assessment. The 197 paired oscillometry-PFTs were collected from 2 different clinical studies: (1) in newly transplanted double lung transplant recipients to monitor graft function post-transplant; and (2) in patients of hematopoietic stem cell transplantation to monitor for pulmonary graft-versus-host disease and bronchiolitis obliterans. The 2 study cohorts provided a broad range of medical diagnoses, sex, and age distribution (Table 1).
Demographics of Both HSCT and LTx Oscillometry Studies
The initial QA/QC identified a number of QC issues. First, assessment of the oscillometry tests identified 10 tests (5.08%) to be invalid, with a coefficient of variation > 15%. These tests were subsequently removed from the data analysis of the original research study (Table 2). Second, it was evident that biometric information, particularly weight, was not being updated at each visit. Because the majority of the study subjects were lung transplant patients, who often have dramatic weight changes over short periods of time due to systemic corticosteroid treatment, this was a critical oversight. Lastly, biological calibrations were reviewed and found to be occurring irregularly (ie, every 3 months).
Invalid Oscillometry Tests*
After the initial QA/QC audit, the SOP (see the supplementary materials at http://www.rcjournal.com) was revised in response to the identified concerns. Specifically, we increased the minimum number of recordings from 3 to 4, introduced a rest period of at least 3 min when a subject is brought to the testing station before conducting the oscillometry test, and reminded the research personnel to update subject weight at each visit. In addition to the above revisions, the 30-s rest in between each recording was introduced by our QA/QC personnel. A QC checklist was developed following the ERS guidelines17 and the Thorasys Tremoflo user manual to ensure that tests were acceptable and reproducible (see the supplementary materials at http://www.rcjournal.com). Good clinical practice and QA criteria (eg, correct input of demographic information) were also introduced by our QA/QC personnel in the QC checklist. The 2 research personnel underwent additional oscillometry training on April 9, 2018. In response, we also developed a 12-h training program that takes place for 4 h/d over 3 consecutive days; this training covers background knowledge of lung physiology, a review of ATS/ERS guidelines for oscillometry, medical professionalism, basic interpretation of PFTs, QC, and interpretations of oscillometry for incoming students and research personnel (see the supplementary materials at http://www.rcjournal.com). Biological calibrations were added to the monthly testing schedule for all research personnel.
Full implementation of the new SOP and the QC checklist were formally deployed on June 28, 2018, and September 13, 2018, respectively. A summer research student who was hired in May 2018 and 5 project students who were hired in September 2018 underwent the same oscillometry training course prior to subject testing. The initial BioQCs performed by 4 different healthy individuals demonstrated no statistical difference between resistance at 5 Hz, reactance at 5 Hz, and the reactance area in the 2 Tremoflo devices in our laboratory.
From April 2019 to June 2019, subject recruitment in both research studies accelerated, leading to an increase in the number of weekly oscillometry tests to an average of 45. As a result, the frequency of informal audits increased gradually from monthly to biweekly. By September 2018, we began formal weekly audits to keep up with the growing pace of oscillometry testing. Feedback was given verbally to individual operators at every opportunity.
Between April 9, 2018 and June 30, 2019, a total of 1,930 oscillometry-PFT paired tests were collected. QA/QC of these 1,930 tests identified only 3 tests (0.16%) to be unacceptable due to a coefficient of variation > 15% (Table 3). These results were significantly different from the initial audit (P < .001).
Comparison of Oscillometry Tests Acceptability
Since the introduction of oscillometry in October 2017 to June 2019, a total of 2,127 paired oscillometry-PFT tests were collected from the 2 clinical studies (Table 4). Of these, 13 paired tests (0.6%) were removed from data analysis due to invalid oscillometry measurements, with a coefficient of variation > 15%.
Comparison of 2,114 Combined Oscillometry-PFT Paired Results
Discussion
Our audits over the course of 14 months revealed that investments in the training of personnel, establishment of a clear SOP, and the implementation of a QC checklist play crucial roles in ensuring the QA/QC of oscillometry, a PFT tool that was introduced into the laboratory < 2 y ago. Our findings are consistent with those reported by Enright and colleagues,25-26 who reported that factors influencing acceptability and reproducibility of the spirometry test are small if the technician is well trained and experienced. The findings of Enright et al26 are based on the review of the quality of spirometry conducted in 13,599 subjects who participated in the World Trade Center Worker and Volunteer Medical Screening Program. Indeed, the number of unacceptable oscillometry tests dropped significantly from April 9, 2018, to June 30, 2019, despite significantly higher volumes of testing and increasing numbers of different personnel conducting the studies compared to the period between October 17, 2017, and April 6, 2018. We believe that our observations are important for the burgeoning number of lung function laboratories that plan to introduce oscillometry as a standard part of lung function testing.
There are several issues beyond our control. Operators of the devices are not able to alter the manufacturer-set calibration settings of the device. The calibration test load used to perform the daily calibration of the Tremoflo device also has limitations. The calibrator has a reference test load at 2.0 cm H2O × s/L, although the ERS recommendation stipulates a magnitude of 15.0 cm H2O × s/L.17 Over the course of > 2,000 tests, we observed a wide range of respiratory resistance and respiratory reactance values, from 0.9 cm H2O × s/L to 14.8 cm H2O × s/L and from −0.1 cm H2O × s/L to −12.7 cm H2O × s/L, respectively.
The software also had factory settings that were not accessible to the operator to determine whether there are inherent errors. Issues relating to proprietary settings of commercial devices for the measurement of specific airway resistance were reported by Poorisrisak et al.27 They examined the accuracy of specific airway resistance at 6 centers and identified software errors in the manufacturer’s settings that were inaccessible to the user. Routine mechanical calibration did not detect these errors. Hence, Poorisrisak et al27 suggested the performance of BioQC to verify and validate these tests measurements.
BioQC is often used to validate testing equipment and procedures. Due to the frequency of testing needed, BioQC subjects should be readily available for evaluation and must be healthy with no known lung disease. Hence, BioQC subjects are usually individuals who work in the pulmonary function laboratory. Currently, no BioQC guidelines exist for oscillometry. We have adapted existing guidelines for PFTs by implementing routine oscillometry BioQC for all personnel involved in the conduct of oscillometry testing monthly at our center. We also cross-validate the measurements between the 2 Tremoflo devices in the lab. Oscillometry BioQC was performed to overcome the limitations of the proprietary software algorithms, device-present calibration setting, and the test load. As oscillometry becomes more frequently used in the clinical setting, we recommend that this modality be included as part of the standard pulmonary function laboratory QA/QC procedures. We also suggest that this modality be included in the next version of the international guidelines for conduct of oscillometry.
Although this was a single-site study that evaluated performance of one commercial device, our study population was well balanced with respect to sex and had a broad range of age and disease types. We have not tested our protocol across multiple laboratory sites nor with different devices. However, we anticipate that our protocol, like guidelines for the performance of conventional PFT, is applicable regardless of the make of the equipment.
Conclusions
The relative ease of performing oscillometry should not offer a false sense of assurance. Our results indicate that formal and proper training of personnel plays an important role in the quality of all pulmonary function testing using all modalities, whether it be spirometry, plethysmography, or oscillometry. Weekly audits of oscillometry tests, immediate feedback, and routine biological calibrations are critically important to ensure that tests are acceptable and reproducible. A standard operating procedure and quality assurance checklist are also required to improve the oscillometry test as a tool in the pulmonary function laboratory. The addition of these procedures to the current guidelines for performing oscillometry will lead to significant improvements, enabling users to confidently perform oscillometry in a clinical setting and ensure that test measurements are collected accurately and precisely.
ACKNOWLEDGMENTS
We thank Lauren Day and all of the registered cardiopulmonary technologists at Toronto General Pulmonary Function Laboratory for providing space to conduct the study, as well as members of Dr Chung-Wai Chow’s laboratory for the collection of oscillometry data. We also thank Honami Ota for her demonstration of oscillometry test in our standard operating procedures.
Footnotes
- Correspondence: Joyce KY Wu RCPT(P), University Health Network, 585 University Ave, 10N1015, Toronto, ON, Canada M5G 2N2. E-mail: joyce.wu{at}uhn.ca
This work was supported by a Pettit Block Term Grant, the Division of Respirology at the University of Toronto, the Princess Margaret Foundation, the Friends for Life Fund, and the Ontario Thoracic Society Grant in Aid. Dr Hantos is supported by Hungarian Scientific Research Fund Grant K128701. The authors have disclosed no conflicts of interest.
Supplementary material related to this paper is available at http://www.rcjournal.com.
- Copyright © 2020 by Daedalus Enterprises
