Research ArticleOriginal Research

An Interrater Reliability Study of Pulmonary Function Assessment With a Portable Spirometer

Hao Zhang, Li Li, Daozhen Jiao, Yanjie Yang, Chunhong Pan, Ling Ye, Sulan Wei and Meiling Jin
Respiratory Care May 2020, 65 (5) 665-672; DOI: https://doi.org/10.4187/respcare.07116

Abstract

BACKGROUND: In this study, we aimed to validate the agreement between pulmonary function measurements obtained with a portable spirometer and measurements obtained with conventional spirometry in Chinese pediatric and adult populations.

METHODS: Pulmonary function testing was performed to evaluate subjects enrolled at Shanghai Zhongshan Hospital (n = 104) and Shanghai Children's Medical Center (n = 103). The portable spirometers and conventional devices were applied to each subject with a 20-min quiescent period between each measurement. Pulmonary function parameters of FVC, FEV1, peak expiratory flow, maximum expiratory flow at 25%, 50%, and 75% of FVC (MEF25, MEF50, and MEF75, respectively), and FEV1/FVC% were compared with intraclass correlation and Bland-Altman methods.

RESULTS: A satisfactory concordance of pulmonary function was observed between spirometry measurements obtained with portable versus conventional spirometers. Intraclass correlation indicated excellent reliability (>0.75) for all pulmonary function indicators in pediatric and adult subjects. Significant positive correlations of all variables measured with different spirometers were observed (all P < .001). No significant bias was observed in either group, although limits of agreement varied. Funnel effects were observed for peak expiratory flow in pediatric subjects and for FVC, FEV1, MEF50, and MEF25 in adult subjects.

CONCLUSIONS: The portable spirometer is an alternative to the conventional device for the measurement of pulmonary function. Compared with the conventional device, the portable spirometer is expected to provide convenient, operational, and financial advantages.

Introduction

Pulmonary function tests (PFTs) are routinely performed to evaluate respiratory function in patients with pulmonary disorders.1 In children, PFTs are also used to assess growth. The results of PFTs imply the type and severity of disease, the effectiveness of treatment, and possible clinical outcomes.1–4 Spirometry is an important measurement tool in PFT. Spirometric results include multiple variables, such as FVC, FEV1, peak expiratory flow (PEF), and maximum expiratory flow at 25%, 50%, and 75% of forced exhalation (MEF25, MEF50, and MEF75, respectively). However, in developing countries, the use of conventional volume-sensing spirometers is generally limited to teaching or comprehensive hospitals due to the cost and complex nature of the operating system. Early screening for respiratory disease and expedient diagnosis are difficult to achieve in most primary hospitals.5 For example, approximately 80% of subjects with COPD were diagnosed with moderate or severe disease, possibly due to a delayed PFT.4–6 Traditional spirometers also have several other disadvantages, including risks associated with cross-contamination, difficulty in cleaning the device, and challenges with use among disabled patients.7

Point-of-care testing has recently been integrated into the health care system and can provide results more quickly through the use of portable, user-friendly devices. Point-of-care testing has been reported to be associated with improved patient outcomes.8 The Yue Cloud spirometer (Shanghai Sonmol Internet Technology, China) is a portable device that may be used to accurately and easily measure of lung function (Fig. 1). When connected to a Windows operating system, this lightweight spirometer can be used to evaluate numerous indices of airway function. Moreover, its flow sensor converts air flow directly into air pressure, thus decreasing the need for filters and any associated risk for cross-contamination. This user-friendly system shortens the clinicians learning curve, which is convenient for clinicians in primary hospitals who seek to screen, diagnose, and monitor patients. In Western countries, the portable spirometer has demonstrated consistency that is similar to that of traditional pulmonary function equipment in the determination of various parameters.9–13 However, scant evidence exists in relation to the reliability of portable spirometry for evaluation of Chinese patients.14

Fig. 1.
Fig. 1.

The portable spirometer. This lightweight portable spirometer is composed of a disposable mouthpiece and a transducer with an internal flow sensor converting air flow directly into air pressure. The unit connects to a mini-screen or a computer via a USB cable. Courtesy Shanghai Sonmol Internet Technology.

In this study, we sought to validate the agreement of measurements obtained with portable versus traditional spirometers in a Chinese cohort. To fully characterize the reproducibility of lung function parameters for the portable spirometer, we enrolled both pediatric and adult populations.

QUICK LOOK

Current knowledge

Pulmonary function tests play an important role in the assessment of respiratory conditions in both pediatric and adult patients. Conventional spirometers are generally not available in primary hospitals for an early screening of respiratory dysfunction. Portable spirometers serve as an alternative to traditional pulmonary function equipment in the determination of various parameters in Western countries.

What this paper contributes to our knowledge

The reliability of the portable spirometer for the measurement of pulmonary function was validated in pediatric and adult subjects. This portable spirometer may provide convenient, operational, and financial advantages for early detection of respiratory disorders.

Methods

Portable Spirometer

The Yue Cloud portable spirometer, weighing only 138 g, is a small handheld device consisting of a pressure sensor and a digital display. Instead of a conventional mouthpiece and hose, this spirometer has a spirette that is inserted through the body of the spirometer. The spirometer can be used alone or connected to a computer with software that displays results in real time.

Unlike traditional spirometers, the Yue Cloud spirometer uses a differential pressure sensor to measure air flow. Air flow through the spirette is converted to a pressure measurement. This approach eliminates errors caused by variables such as temperature, gas composition, and humidity. Its standardized quality-control system is suitable for various situations. Use of this portable spirometer in connection with a web workstation allows for remote diagnosis and prescription (Fig. 1).

Subject Population

A total of 207 consecutive subjects who underwent PFT at Shanghai Zhongshan Hospital (n = 104 adult subjects) and Shanghai Children's Medical Center (n = 103 pediatric subjects) between February and July 2018 were enrolled in this study. All subjects enrolled in the study received standardized instructions for how to use the spirometer. All subjects underwent 2 sets of PFTs: the first using the Jaeger MasterScreen IOS (Vyaire Medical, Höchberg, Germany); the second using the Yue Cloud spirometer. The study was approved by the Ethics Committee of Shanghai Zhongshan Hospital and Shanghai Children's Medical Center. Informed consent was obtained for all subjects.

Measurement

Measurements of pulmonary function were carried out by the same technician on the same instruments at the pulmonary function laboratory. Calibration and measurement were strictly in compliance with the 2005 American Thoracic Society/European Respiratory Society guidelines.4 Uniform predicted values were applied in both systems. The quiescent period between measurements was 20 min before each measurement. Results obtained with the portable spirometer were automatically displayed using the accompanying software and printed out. Measurements of pulmonary function obtained using the Jaeger lung function instrument were evaluated by the technician based on curve morphology. Spirometry variables including vital capacity, FVC, FEV1, FEV1/FVC, PEF, MEF75, MEF50, and MEF25 were reported. We defined the categories of lung function impairment as follows: 1) obstructive impairment: FEV1/FVC < 0.70 for adult subjects and FEV1/FVC < 0.80 for subjects 5–18 y of age; 2) restrictive impairment: FEV1/FVC ≥ 0.70 and FVC < 80% predicted for adult subjects and FEV1/FVC ≥ 0.80 and FVC < 80% for pediatric subjects. Severity was classified as mild, moderate, or severe, according to the FEV1 percentage of predicted > 70%, 50–69%, or ≤ 49% for adult subjects and 60–79%, 40–59%, or ≤ 39% for pediatric subjects, respectively. The predicted values of FEV1 and FVC were calculated based on the prediction equations from the European Respiratory Society.15

Statistical Analysis

Categorical data are expressed as n (%), and continuous data are expressed as mean ± SD. Lung function parameters including vital capacity, FVC, FEV1, FEV1/FVC, PEF, MEF75, MEF50, and MEF25 were compared between groups. Concordance was assessed using the intraclass correlation coefficient (ICC)16,17 and Bland-Altman analysis.6 An ICC close to 1 indicates high similarity among measurements. The Pearson correlation was used to analyze relationships between variables. The overall agreement in diagnosis of lung function impairment between devices was assessed using the Cohen kappa test. The kappa coefficient indicating the strength of diagnosis agreement was calculated and qualified on the basis of its magnitude: 0.4–0.6, moderate agreement; 0.6–0.8, substantial agreement; and 0.8–1.0, almost perfect agreement.18 All statistical analysis was conducted with SPSS 24.0 (IBM Corp., Armonk, New York). A P value < .05 was considered statistically significant.

Results

Demographic and Clinical Characteristics

The average ages of adult subjects and pediatric subjects were 56.7 ± 13.5 y and 8.4 ± 2.5 y, respectively. Male subjects accounted for 61.6% of the adult study population and 64.1% of the pediatric population. The average heights of the adult subjects and pediatric subjects were 165.4 ± 8.3 cm and 134.5 ± 15.6 cm, respectively. Average body weights for adult subjects and pediatric subjects were 64.7 ± 11.5 kg and 32.9 ± 11.6 kg, respectively. Subjects with abnormal PFT results accounted for 28.2% of the pediatric population and 11.5% of the adult population (Table 1). Baseline measurements were obtained with the Jaeger MasterScreen IOS.

View this table:
Table 1.

Baseline Characteristics of the Study Population

ICC Analysis

No systematic rater differences in measured pulmonary function values were found. The ICCs in pediatric subjects showed that PFTs including FVC, FEV1, FEV1/FVC, PEF, MEF75, MEF50, and MEF25 were all > 0.75, indicating that the parameters measured by the portable spirometer were reliable in terms of within-person variability. Similarly, the ICCs for all measured pulmonary indicators in adults ranged from 0.874 for FEV1/FVC to 0.955 for MEF75 (Table 2).

View this table:
Table 2.

Intraclass Correlation Coefficients for Pulmonary Function Parameters

Bland-Altman Analysis

In pediatric subjects, no significant bias was found in the measurements of FVC, FEV1, PEF, MEF75, MEF50, or MEF25, except that FEV1/FVC values tended to be lower when measured with the portable spirometer compared to the conventional system (mean 0.61, 95% CI −12.64 to 11.42). This trend primarily reflects the participation of male pediatric subjects when male and female subjects were evaluated respectively (male pediatric subjects: mean 1.09, 95% CI −11.47 to 13.64; female pediatric subjects: mean −0.24, 95% CI −11.24 to 10.76). In adult subjects, there was no significant bias in any of the measured spirometry variables. No gender difference between the 2 sets of measurements was observed in adults. Differences and 95% limits of agreement (LoA) are displayed in Figure 2 and Figure 3 (see the supplementary materials at http://www.rcjournal.com).

Fig. 2.
Fig. 2.

Bland-Altman plots with 95% limits of agreement summarizing the results of pulmonary function tests in pediatric subjects. PEF = peak expiratory flow; MEF = maximum expiratory flow at 25%, 50%, and 75% of FVC (MEF25, MEF50, and MEF75, respectively).

Fig. 3.
Fig. 3.

Bland-Altman plots with 95% limits of agreement summarizing the results of pulmonary function tests in adults. PEF = peak expiratory flow; MEF = maximum expiratory flow at 25%, 50%, and 75% of FVC (MEF25, MEF50, and MEF75, respectively).

Heteroscedasticity arose in the PEF data (P = .039) collected for pediatric subjects and in measurements of FVC (P = .01), FEV1 (P = .033), MEF50 (P < .001), and MEF25 (P < .001) in adult subjects. Transformation of these data into natural logarithms reduced the degree of heteroscedasticity (PEF: P = .16; FVC: P = .54; FEV1: P = .56; MEF50: P = .40; MEF25: P = .16) (see the supplementary materials at http://www.rcjournal.com).

Kappa Coefficient

The adult subjects with obstructive, restrictive, or mixed lung dysfunction accounted for 11.5% (12 of 104 subjects) using data measured with the conventional spirometer, whereas the percentage was 16.3% (17 of 104 subjects) when diagnosed with the portable spirometer. The kappa coefficient reflecting the agreement is 0.807.

In pediatric subjects, 28.2% (29 of 103 subjects) and 25.2% (26 of 103 subjects) were diagnosed with lung dysfunction with the conventional or with the portable spirometer, respectively. The kappa coefficient was 0.702.

Discussion

This study evaluated the reliability of a portable spirometer by comparing the concordance between measurements obtained with the portable spirometer and with a conventional spirometer. The spirometry parameters examined included vital capacity, FVC, FEV1, FEV1/FVC, PEF, MEF75, MEF50, and MEF25. Our results indicate that PFT measurements obtained with the portable spirometer were closely correlated with those obtained using the conventional spirometer. A consistently good ICC > 0.75 was obtained for all measured variables. The results of Bland-Altman analysis for measurements obtained using the portable spirometer showed an acceptable level of bias. The diagnostic agreement as reflected by the kappa statistic revealed good agreement, especially in the adult subjects.

The ICC values for pediatric subjects and adult subjects reflected good to excellent reliability. In the pediatric subjects as well as the adult subjects, FVC and FEV1 values were > 0.90. For other variables, ICC values were slightly lower but remained > 0.75. ICC values between 0.75 and 0.90 indicate satisfactory reliability, and values > 0.90 indicate excellent reliability.17 Our ICC results suggest that the differences between measurements derived primarily from individual differences between subjects rather than from differences between the spirometers.

The results of Bland-Altman analyses revealed that only 1–7% of measurements were beyond the LoA for all measured PFT variables. Close correlation with statistical significance was confirmed for all spirometric variables. However, we noticed slightly lower mean values for FEV1, PEF, and MEF75 in both cohorts, which was in line with previous findings.19–22 The order of detection may affect the results of PFT. In this study, conventional spirometric measurements were used as the first set. Subjects were asked to exhale repeatedly to obtain ideal measurement results. Although all subjects had 20 min resting time between measurements with different spirometers, fatigue may still be inevitable, particularly for elderly participants.

The LoA is a key indicator for assessing interchangeability between methods. In this study, although mean between-measurement differences in FVC and FEV1 were all < 110 mL, variation in LoA was greater. This phenomenon was observed in previous studies of portable spirometers.20,21 However, there is little evidence regarding LoA acceptability for PFT parameters. In previous studies on COPD, an LoA of 500 mL was considered acceptable for FVC, and an LoA of 350 mL was considered acceptable for FEV1.9,23 LoA for FVC and FEV1 in pediatric subjects were acceptable, but greater LoA for both variables were observed in adult subjects. We speculate that the learning effect, order of measurement, and exhalation end point may have affected the LoA.

It is noteworthy that some funnel effects were observed in relation to PEF in the pediatric subjects and in relation to FVC, FEV1, MEF50, and MEF25 in the adult subjects. The use of portable spirometry may therefore be associated with an underestimation of these variables.

Our findings confirmed generally good agreement between measurements obtained with a portable spirometer and those obtained with a conventional spirometer. Advantages of the portable spirometer include portability and ease of use. A prompt assessment of PFT is therefore feasible for screening purposes in the general population and in emergency situations. One recent, large-scale, prevalence study of COPD in China found that almost 100 million Chinese adults had COPD. Most patients with COPD were unaware of their condition, and only 12% of them had undergone PFT prior to the study.24 With the application of portable spirometers, clinicians at primary hospitals will be able to easily perform PFT and COPD screenings for high-risk individuals.4,5 For preschool-age children, our results confirmed that they were able to comply with the portable maneuvers to produce consistent PFT curves.12 The portable spirometer has the potential to improve our management of pulmonary diseases for children as well as the assessment of growth because spirometric pulmonary function was related to height.25

This study had several limitations. First, the portable spirometer, like conventional spirometers, cannot be used to measure diffusion capacity. Second, we did not analyze the difference between spirometers in subcategorized subjects with different disease severity according to PFT results because of the limited sample size. Third, we did not randomize the device with which the subjects performed first, which might affect the results. Large-scale validation studies should be conducted in the future.

Conclusions

Satisfactory concordance was observed between PFT measurements obtained with the portable spirometer and PFT measurements obtained with conventional equipment. The portable spirometer is an ideal alternative to the conventional device for measurements of pulmonary function. Compared with the conventional device, the table spirometer may provide operational and financial advantages. This portable spirometer is primarily indicated for use in screening the general population and following up lung function in patients with respiratory diseases.

Acknowledgments

The authors thank the technicians at the pulmonary function laboratories.

Footnotes

  • Correspondence: Meiling Jin MD, Department of Respiratory Medicine, Zhongshan Hospital Affiliated with Fudan University, No. 180 Fenglin Rd, Shanghai 200032, China. E-mail: mljin118{at}163.com

  • Drs Zhang and Li are co-first authors.

  • This study was supported by the National Key Research & Development (R&D) Program of China (2016-YFC1304000, 2016-YFC1304002), the Shanghai Top-Priority Clinical Key Disciplines Construction Project (2017ZZ02013), and the Key Project of Shanghai Science and Technology Committee, Department of Biomedicine (164 1195 0800, to Dr Zhang). The authors have disclosed no conflicts of interest.

  • Supplementary material related to this paper is available at http://www.rcjournal.com.

References

  1. 1.↵
    1. Ulmer WT
    . Lung function–clinical importance, problems, and new results. J Physiol Pharmacol 2003;54(Suppl 1):11–13.
    OpenUrl
  2. 2.
    1. Takahashi Y,
    2. Suzuki S
    . Preoperative pulmonary function testing and postoperative complications. J Thorac Dis 2018;10(Suppl 33):S3840–S3842.
    OpenUrl
  3. 3.
    1. Krishnan JK,
    2. Martinez FJ
    . Lung function trajectories and chronic obstructive pulmonary disease: current understanding and knowledge gaps. Curr Opin Pulm Med 2018;24(2):124–129.
    OpenUrl
  4. 4.↵
    1. Buffels J,
    2. Degryse J,
    3. Heyrman J,
    4. Decramer M
    . Office spirometry significantly improves early detection of COPD in general practice: the DIDASCO Study. Chest 2004;125(4):1394–1399.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Díaz-Lobato S,
    2. Mayoralas S
    . Underuse of spirometry in primary care. Chest 2004;126(5):1712.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Peng X,
    2. Huang M,
    3. Zhao W,
    4. Yuan Y,
    5. Li B,
    6. Ye Y,
    7. et al
    . Delayed diagnosis is associated with greater disease severity of chronic obstructive pulmonary disease. Nan Fang Yi Ke Da Xue Xue Bao 2018;38(12):1448–1452.
    OpenUrl
  7. 7.↵
    1. Hankinson JL
    . Beyond the peak flow meter: newer technologies for determining and documenting changes in lung function in the workplace. Occup Med 2000;15(2):411–420.
    OpenUrlPubMed
  8. 8.↵
    1. Price CP
    . Point of care testing. BMJ 2001;322(7297):1285–1288.
    OpenUrlFREE Full Text
  9. 9.↵
    1. Liistro G,
    2. Vanwelde C,
    3. Vincken W,
    4. Vandevoorde J,
    5. Verleden G,
    6. Buffels J
    . Technical and functional assessment of 10 office spirometers: a multicenter comparative study. Chest 2006;130(3):657–665.
    OpenUrlCrossRefPubMed
  10. 10.
    1. Bambra G,
    2. Jalota L,
    3. Kapoor C,
    4. Mills PK,
    5. Vempilly JJ,
    6. Jain VV
    . Office spirometry correlates with laboratory spirometry in patients with symptomatic asthma and COPD. Clin Respir J 2017;11(6):805–811.
    OpenUrl
  11. 11.
    1. Suzuki S,
    2. Takahashi Y
    . Office-based spirometry to stratify the risk of postoperative complications. Ann Transl Med 2018;6(Suppl 1):S59.
    OpenUrl
  12. 12.↵
    1. Mortimer KM,
    2. Fallot A,
    3. Balmes JR,
    4. Tager IB
    . Evaluating the use of a portable spirometer in a study of pediatric asthma. Chest 2003;123(6):1899–1907.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Ezzahir N,
    2. Leske V,
    3. Peiffer C,
    4. Trang H
    . Relevance of a portable spirometer for detection of small airways obstruction. Pediatr Pulmonol 2005;39(2):178–184.
    OpenUrlPubMed
  14. 14.↵
    1. Zou K,
    2. Ni C,
    3. He Z,
    4. Zhang Y,
    5. Ma Z,
    6. Song J,
    7. Zhang K
    . [Reproducibility of a portable spirometer based on differential pressure sensor]. Zhongguo Yi Liao Qi Xie Za Zhi 2013;37(1):30–32,48.
    OpenUrl
  15. 15.↵
    1. Quanjer PH,
    2. Tammeling GJ,
    3. Cotes JE,
    4. Pedersen OF,
    5. Peslin R,
    6. Yernault JC
    . Lung volumes and forced ventilatory flows. Eur Respir J 1993;6(Suppl 16):5–40.
    OpenUrlFREE Full Text
  16. 16.↵
    1. Yen M,
    2. Lo LH
    . Examining test-retest reliability: an intra-class correlation approach. Nurs Res 2002;51(1):59–62.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Lee J,
    2. Koh D,
    3. Ong CN
    . Statistical evaluation of agreement between two methods for measuring a quantitative variable. Comput Biol Med 1989;19(1):61–70.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. McGinn T,
    2. Wyer PC,
    3. Newman TB,
    4. Keitz S,
    5. Leipzig R,
    6. For GG
    , Evidence-Based Medicine Teaching Tips Working Group. Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic). CMAJ 2004;171(11):1369–1373.
    OpenUrlFREE Full Text
  19. 19.↵
    1. Dirksen A,
    2. Madsen F,
    3. Pedersen OF,
    4. Vedel AM,
    5. Kok-Jensen A
    . Long-term performance of a hand held spirometer. Thorax 1996;51(10):973–976.
    OpenUrlAbstract/FREE Full Text
  20. 20.↵
    1. Malmberg LP,
    2. Hedman J,
    3. Sovijarvi AR
    . Accuracy and repeatability of a pocket turbine spirometer: comparison with a rolling seal flow-volume spirometer. Clin Physiol 1993;13(1):89–98.
    OpenUrlPubMed
  21. 21.↵
    1. Nishimura K,
    2. Nakayasu K,
    3. Kobayashi A,
    4. Mitsuma S
    . Case identification of subjects with airflow limitations using the handheld spirometer “Hi-Checker”: comparison against an electronic desktop spirometer. COPD 2011;8(6):450–455.
    OpenUrl
  22. 22.↵
    1. Brouwer AF,
    2. Roorda RJ,
    3. Brand PL
    . Comparison between peak expiratory flow and FEV(1) measurements on a home spirometer and on a pneumotachograph in children with asthma. Pediatr Pulmonol 2007;42(9):813–818.
    OpenUrlPubMed
  23. 23.↵
    1. Enright PL,
    2. Connett JE,
    3. Kanner RE,
    4. Johnson LR,
    5. Lee WW
    . Spirometryin the Lung Health Study: II. Determinants of short-term intraindividual variability. Am J Respir Crit Care Med 1995;151(2 Pt 1):406–411.
    OpenUrlPubMed
  24. 24.↵
    1. Fang L,
    2. Gao P,
    3. Bao H,
    4. Tang X,
    5. Wang B,
    6. Feng Y,
    7. et al
    . Chronic obstructive pulmonary disease in China: a nationwide prevalence study. Lancet Respir Med 2018;6(6):421–430.
    OpenUrl
  25. 25.↵
    1. Eigen H,
    2. Bieler H,
    3. Grant D,
    4. Christoph K,
    5. Terrill D,
    6. Heilman DK,
    7. et al
    . Spirometric pulmonary function in healthy preschool children. Am J Respir Crit Care Med 2001;163(3 Pt 1):619–623.
    OpenUrlCrossRefPubMed
Back to top

In this issue

 

Print
Download PDF
Article Alerts
Email Article
Citation Tools

Jump to section

Related Articles

Cited By...

Keywords