Abstract
BACKGROUND: Chest physiotherapy has been reported to be beneficial in specific clinical contexts, yet it carries a risk of potential serious adverse events with little benefit in other patients. Therefore, identifying and limiting airway clearance therapies to patients with the greatest potential benefit and least risk is clinically relevant and important. This study aims to validate the Airway Clearance and Expansion Index (ACE-I) for the serial assessment of hospitalized pediatric patients with impaired airway clearance and to establish reliability in score acquisition across a range of pediatric respiratory disease states.
METHODS: Content validity of the category importance and category choices was assessed via a survey of well-established pediatric pulmonary and critical care physicians, as well as respiratory therapists (RTs). Inter-rater reliability testing was performed on hospitalized children from October 2016 through April 2017 and analyzed using a one-way random effects intraclass correlation.
RESULTS: 51 providers (24 of 37 physicians and 27 of 92 RTs) responded to the survey. Agreement was defined as any score of 6 or greater out of 10 on a scale of 1–10. The total ACE-I scale content validity index (S-CVI) scores for category importance and category choices for physicians were 1 and 0.75, respectively, and for RTs the scores were 0.75 and 0.75, respectively. 172 subjects were scored by multiple raters, resulting in an excellent overall intraclass correlation coefficient of 0.77 (95% CI 0.71–0.83) and the following component scores: cough, 0.72 (95% CI 0.64–0.79); breath sounds, 0.54 (95% CI 0.43–0.64); chest radiograph findings, 0.84 (95% CI 0.79–0.88); and secretions 0.85, (95% CI 0.81–0.89).
CONCLUSIONS: The ACE-I score addresses and quantifies 4 components of the respiratory assessment that RTs and pediatric physicians deem important in identifying patients who have impaired airway clearance and might benefit from airway clearance and expansion therapies. In addition, the ACE-I score had excellent inter-rater reliability and clinical feasibility within our single institution.
Introduction
Natural airway clearance depends on normal ciliary function, adequate cough strength, unobstructed airways, and appropriate production and composition of mucus.1 Often one or more of these functions become impaired in a wide variety of pediatric respiratory illnesses. In these settings, airway clearance becomes impaired and a wide variety of techniques and modalities, including postural drainage, percussion, vibration of the chest wall, and coughing assistance with a multitude of devices are often trialed to aid in mucus mobilization and expectoration.1 These are often referred to in the literature under the umbrella term “chest physiotherapy” and as a group will be referred to as “airway clearance and expansion therapies” in this paper. Following this rationale, chest physiotherapy use has been reported in the literature for the treatment of pediatric bronchiolitis,2 cystic fibrosis,3 non-cystic fibrosis bronchiectasis,4 intubated pediatric patients,5 asthma,6 and pneumonia.7
There is a plethora of literature that shows clinical benefit of chest physiotherapy when targeting appropriate therapies to specific patient populations. These benefits include improved lung compliance in paralyzed mechanically ventilated children,5 improved airway clearance in hospitalized patients with asthma,8 decreased atelectasis scores,9 decreased hospital length of stay for patients with bronchiolitis,10 and prevention of postextubation atelectasis in children with neuromuscular diseases.11 On the contrary, several published studies discourage the use of chest physiotherapy for the routine care of intubated children,12 children with asthma,13 patients with bronchiolitis,14 and patients with uncomplicated pneumonia7,15 due to a lack of evidence for benefit. In addition, several studies have shown a risk of adverse events including desaturation,16 increased oxygen demand, increased heart rate, and increased intracranial pressure.12 Despite these recommendations, chest physiotherapy continues to be a common practice in the pediatric ICU and in hospitalized patients. A recently published multicenter study reported that nearly 50% of pediatric ICU subjects received chest physiotherapy during their stay.17
A multitude of respiratory scores currently exist in pediatrics to identify particular patient pathological phenotypes, to predict the need for escalating care, to provide succinct comparison of respiratory symptoms before and after an intervention, and to allow tracking of respiratory status throughout a hospital course. However, none of the existing scores specifically address symptomology of impaired airway clearance. Development of a simple, numeric, in-patient clinical score to identify patients with persistent impaired airway clearance is clinically relevant and important. Ideally, this tool would allow care providers to track a patient's airway clearance ability across time after initiation of therapies, which could facilitate clinical trials or observational comparative effectiveness studies, and, if proven to be important, may eventually guide clinical management.
To provide a clinical measure that reflects a given patient's overall airway clearance status, we have devised a simple and easy-to-apply clinical index: the Airway Clearance and Expansion Index (ACE-I). We provide evidence of the content validity of this scale through literature review and survey results of experienced pediatric pulmonary and critical care physicians and respiratory therapists (RTs). In addition, we provide evidence of inter-rater reliability in a large cohort of hospitalized pediatric patients with a variety of respiratory diseases.
QUICK LOOK
Current knowledge
The efficacy and utility of homogenously applied airway clearance and expansion therapies for children hospitalized with respiratory distress is both contradictory and controversial. No clinical scores have been developed for hospitalized pediatric patients to uniformly identify impaired airway clearance in patients receiving therapies.
What this paper contributes to our knowledge
The Airway Clearance and Expansion Index (ACE-I) categorized secretions, chest radiograph findings, cough quality, and breath sounds into a single numeric score. The overall score showed good content and criterion validity for impaired secretion clearance with excellent inter-rater reliability and clinical feasibility.
Methods
This data collection was deemed to be in line with the attributes of a larger quality-improvement project, and therefore human subject participation and data collection were covered under IRB-041-02 at Nationwide Children's Hospital in Columbus, Ohio.
Instrument Development
The initial proposed tool was patterned off an existing asthma clinical score that included tachypnea for age, supplemental oxygen requirement, air movement, retractions, and wheezing. We added additional categories of: secretions, cough quality, and chest radiograph findings. These were based on support in the literature for the use of airway clearance techniques for inadequate secretion clearance, inadequate cough, and recurrent or persistent atelectasis, respectively. The proposed score underwent many revision and feedback cycles from January 2016 to October 2016. Each iteration contained changes based on small-sample qualitative feedback. This resulted in a twice-daily scoring scheme and the removal of categories for tachypnea for age, oxygen requirement, retractions, and wheezing. The matrix then underwent extensive terminology edits until deemed ready for assessment of content validity and inter-rater reliability testing. The final proposed scoring tool that was analyzed in this study is presented in Table 1.
Index Development
Content validity of the score components and category choices were assessed using a survey of well-established pediatric care providers. Pediatric pulmonary and critical care faculty and pediatric RTs were recruited via an email survey utilizing REDCap (Research Electronic Data Capture) tools hosted at a large pediatric quaternary-care academic center. REDCap is a secure, web-based application designed to support data capture for research studies using a data-protective REDCap tool.18 Survey participants evaluated 2 statements about each of 4 factors and recorded their choices on a 10-point Likert scale:
How important is each category to you in determining a patient's airway clearance therapies and frequency (ie, is that factor important to you when choosing initial airway clearance modality and frequency)?
How accurately do the category choices reflect possible patient characteristics (ie, would you be able to select a choice for most patients)?
Reliability Evaluation
Inter-rater reliability testing was performed on hospitalized children from October 2016 through April 2017 with active airway clearance and expansion-therapy orders, including use of hand percussion, a pneumatic percussion device, a vibratory percussion device, high-frequency chest wall oscillation by airway clearance vest, cough assist, and intrapulmonary percussive ventilation. Two scoring RTs who had not yet cared for a given subject during the current hospitalization observed the subject as the subject's primary RT performed a respiratory assessment and provided any ordered therapies. The 2 scoring RTs then independently logged their score assessments again using REDCap. Each subject could only be assessed twice by a pair of RTs, thus generating a total of 4 scores per hospitalization, and never by the same RT twice. Age, level of respiratory support, current frequency of airway clearance and expansion therapies, and unit of hospitalization were collected for each subject, and the association between each subject and the ACE-I score was assessed as a means of demonstrating criterion validity for all scores.
Statistical Analysis
Results of the content validity survey are described using 2-way bar charts by profession. A respondent answering at or above a threshold of 6 out of 10 was defined as being in agreement with the statement. An item-level content validity index (I-CVI) and a scale-level content validity index (S-CVI) were calculated based on Lynn's methods19 and critical values defined by Ayre and Scally.20 Subject characteristics were presented as frequencies, and median ACE-I score was calculated for each category. Categorical comparisons were performed using the Kruskal-Wallis test. Inter-rater reliability was performed using a 1-way random effects intraclass correlation.21 Total score and component analyses were performed as presented. An intraclass correlation coefficient of 0.40–0.59 was considered fair, 0.60–0.74 was considered good, and > 0.75 was considered excellent.21 Internal validity was assessed using Cronbach alpha. Analysis was performed in STATA 13 (StataCorp, College Station, Texas).
Results
Index Development
Overall, 51 providers responded to the survey; 24 respondents were attending physicians (24 of 37 recipients, for a 65% response rate) from either pulmonary or critical care specialty with a median of 15 y of clinical experience. The remaining 27 respondents were RTs (27 of 92 recipients, for a 29% response rate) with a median of 10 y of clinical experience. Categorical importance survey results with I-CVI are presented in Figure 1, and accuracy of category choices results with I-CVI are presented in Figure 2. S-CVI values for physicians were 1 and 0.75 for category importance and category choices, respectively. S-CVI values for RTs were 0.75 and 0.75 for category importance and category choices, respectively. No category omissions were consistently identified.
Reliability Testing
172 subjects received paired scoring in the inter-rater reliability assessment. Subject characteristics are reported in Table 2, and median total ACE-I score is presented by group. There was a higher median ACE-I score for those receiving higher respiratory support, more frequent airway clearance therapies, and for those in the ICU compared to other units (Table 2). There was no difference in median score by age group. Results of the intraclass correlation assessment and internal validity testing are presented in Table 3. Overall, the total score had an intraclass correlation coefficient of 0.77, with the lowest component score of 0.54 for breath sounds.
Discussion
Our aim was to develop a new clinical tool that reliably identifies and serially measures impaired airway clearance in pediatric patients hospitalized for respiratory distress. Our results demonstrate that the newly developed ACE-I instrument has good content validity, appropriately addressing factors deemed important for the initial identification of impaired airway clearance by experienced pediatric pulmonologists, intensivists, and RTs. We showed good concurrent criterion validity with significant differences in median ACE-I score between subject groups based on the current level of respiratory support, unit of care, and current frequency of airway clearance and expansion therapies. Furthermore, the overall score had excellent inter-rater reliability and was clinically feasible in that it was easily obtained by a trained RT during a standard round of assessments and therapies.
The primary goal of chest physiotherapy is to improve the mobilization of airway secretions. Most studies of the effectiveness of chest physiotherapy to date have been based on homogenous application of therapies to a given diagnosis regardless of differences in specific patient symptomatology.7,12,13,16 Many providers agree that targeted chest physiotherapy should be based on prevailing pathophysiology rather than on general diagnostic classifications.22 Specifically, it has been suggested that “the rationale for treatment should be based on excessive secretions, atelectasis, or abnormal gas exchange.”23 Many pathological processes can lead to impaired airway clearance, and we therefore do not expect that presence of a given characteristic such as atelectasis will be associated with another such as poor cough. We do, however, hope to capture both characteristics in one score. Thus, we intend the tool to function as an index rather than as a scale. In keeping with this goal, our Cronbach alpha values were expectedly low. Our data suggest that the ACE-I tool succinctly provides a standardized, reproducible measurement of these characteristics.
While the clinical utility of the ACE-I instrument has not yet been formally assessed, its potential impact is wide-reaching. It is an attractive tool for implementation in an RT-driven protocol because it creates a common language for providers to discuss a patient's respiratory status and to track changes over time after the initiation of therapies. Lowe et al24 recently demonstrated that use of a care bundle that included a “β-agonist/airway clearance protocol score” was associated with reduced stay, β-agonist therapies, airway clearance therapies, and ventilator days. Although the score was not formally tested for validity and reliability, it contained factors similar to the ACE-I score including cough, secretions, chest radiograph findings, and breath sounds, among others.
Prior studies of chest physiotherapy in hospitalized children were limited by the fact that a diversity of interventions were applied broadly across a diagnostic group (eg, bronchiolitis) rather than targeted to individual subjects with clinical features indicative of impaired airway clearance. The ACE-I score has several potential advantages for use in future research studies because it provides a reliable, scalar measurement of airway clearance ability after application of carefully selected and defined airway clearance modalities or mucolytic therapies at the individual patient level. One critique of the review of chest physiotherapy for bronchiolitis by Roque i Figuls et al14 is that it did not plan for any subgroup analysis or investigation of heterogeneity of effect. Specifically, Roque i Figuls et al14 did not address whether patients with evidence of impaired mucus clearance would fare better with chest physiotherapy, as suggested by Lin and Madikians.25 Cross and Elender26 similarly alluded to the difficultly in assessing for heterogeneity of effects, commenting that “indiscriminate use of manual chest therapy may disguise real benefit in certain circumstances.” The ACE-I tool, if utilized, would provide a standardized, scalar measurement of impairment of airway clearance and provide a basis for the investigation of heterogeneity of effect after the application of therapies. Alternatively, the ACE-I score itself could be utilized as an intermediate outcome in tracking changes in airway clearance ability overtime after the application of different therapies.
The generalizability of these findings is limited by the single-center design. In addition, the in-house training and assessment standards of our respiratory therapists may not be reflective of all pediatric hospitals. Therefore, the high level of inter-rater reliability may not be immediately achieved at other institutions. Our results are not directly comparable to other established pediatric respiratory scores because the ACE-I is the first such instrument to address impaired airway clearance alone. Furthermore, there is no accepted standard measurement of impaired airway clearance, so we have limited ability to assess for convergent validity. Finally, our study does not begin to address the complex issues related to the selection and appropriate delivery of the multitude of modalities currently available; for the sake of simplicity, we included all modalities in the umbrella term of airway clearance therapies. Future studies addressing clinical impact will need to standardize and define explicitly the therapies provided.
Conclusion
The Airway Clearance and Expansion Index (ACE-I) has good content and criterion validity for the identification of hospitalized patients with impaired airway clearance. The overall score has excellent inter-rater reliability and clinical feasibility for patients receiving airway clearance therapies. These results suggest that the ACE-I may be an effective tool in a clinical pathway to reduce clinical variability and target the use of pediatric airway clearance and expansion therapies following future clinical validation. The next step will be to use the ACE-I in a prospective trial to investigate whether it effectively identifies patients more likely to benefit from airway clearance and expansion therapies.
Footnotes
- Correspondence: Brittany Shutes MD, Department of Pediatrics, Division of Pediatric Critical Care, Nationwide Children's Hospital, 700 Children's Drive, Columbus, Ohio 43205.
The authors have disclosed no conflicts of interest.
- Copyright © 2019 by Daedalus Enterprises
References
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.