Abstract
BACKGROUND: We sought to compare the accuracy of a newly developed childhood asthma score (CAS) with routine clinical assessment of respiratory status in children with acute asthma in predicting requirements for bronchodilator nebulization.
METHODS: In this prospective observational study in children 2–18 y old with acute asthma, we evaluated the association between the CAS and routine clinical assessment as well as inter-rater agreement.
RESULTS: The need for bronchodilator nebulization was assessed during 134 episodes of acute asthma in 47 children. Overall, bronchodilators were administered after routine clinical assessment in 74 episodes (55.2%). The median CAS was 2.5 (interquartile range of 2.0–3.0) for subjects who did not receive nebulization and 6.0 (interquartile range of 4.0–7.0) for subjects who did receive nebulization (P < .001). A CAS cutoff score of 4 yielded a sensitivity of 0.91 (95% CI 0.84–0.97) and a specificity of 0.77 (95% CI 0.66–0.87), with a positive predictive value of 0.83 (95% CI 0.75–0.91) and a negative predictive value of 0.87 (95% CI 0.78–0.96). In 79 episodes, the CAS was assessed by 2 independent raters. With a weighted kappa of 0.77, a good inter-rater agreement was obtained.
CONCLUSIONS: Using a cutoff value of 4, the newly developed CAS accurately predicts the requirement for bronchodilator nebulization in children with acute asthma without use of auscultative findings.
Introduction
The clinician's judgment to adjust the frequency of administration of bronchodilators in acute asthma is based on the clinical assessment of the patient's respiratory status. However, this clinical assessment requires physician assessment of auscultative findings. In daily practice, a physician is not always readily available to perform the assessment. By using a standardized respiratory assessment, the in-patient care of children with acute asthma may be improved by reducing variability in the decision-making process. Several clinical scores have been developed and shown to correlate well with the degree of asthma severity and response to treatment.1–7 This makes the different asthma scores attractive for use in clinical pathways. Indeed, it has been demonstrated that the use of an asthma score in a clinical pathway for acute asthma reduced the hospital stay without increased morbidity.8,9
However, the currently available pediatric asthma scores require auscultation of the lungs to score the degree of wheezing or air entry. Auscultation requires adequate training to minimize subjectivity. This implies that all health-care providers should be equally experienced in auscultation of the lungs.
A scoring system that does not include auscultation of the lungs reduces the degree of subjectivity, facilitates communication between different health-care providers, and increases likelihood of appropriate administration of bronchodilators. If such an asthma score correlates sufficiently with routine clinical judgment by a physician, this score could be used by other health-care professionals such as respiratory therapists and nurses.
The objective of this study was to assess the value of a newly developed childhood asthma score (CAS), which does not require lung auscultation, in predicting the need for bronchodilator nebulization in children with acute asthma.
QUICK LOOK
Current knowledge
A standardized respiratory assessment in the in-patient care of children with acute asthma may reduce variability in the decision-making process. Several clinical scores have been developed and shown to correlate well with the degree of asthma severity and response to treatment.
What this paper contributes to our knowledge
A newly developed childhood asthma score consisting of breathing frequency, dyspnea, retractions, and oxygen saturation accurately predicted the requirement for aerosolized bronchodilator treatment in children with acute asthma without use of auscultative findings.
Methods
Between September 2010 and September 2011, we introduced a newly developed CAS as a standard assessment tool for all children 2–18 y old admitted with acute asthma. We chose to exclude patients < 2 y of age because many patients in this age group wheeze recurrently without having a definite diagnosis of asthma. Only children who responded to the first bronchodilator nebulization and were admitted for further treatment with bronchodilators were included. A pool of nurses was instructed to measure the 12-point CAS, a composite score comprising oxygen saturation and 3 physical findings (Table 1). The CAS was adapted from the physical findings in pediatric asthma scores validated previously by Parkin et al1 and Liu et al.3 Oxygen saturation was included in the CAS because this value at presentation in the emergency department has been shown to predict the need for intensive bronchodilator therapy.10,11 The total score ranges from 2 to 12, with a higher score indicating more respiratory distress.
Items of Childhood Asthma Score Per Age Group
During the course of the disease, children were treated according to the local protocol for acute childhood asthma. According to standard care in our hospital, the pediatrician or pediatric resident assessed the patient's respiratory status and determined whether to administer bronchodilators (albuterol plus ipratropium bromide). Children with chronic diseases other than asthma were excluded, as well as patients who required continuous administration of bronchodilators, intravenous magnesium sulfate, or intravenous albuterol.
During each episode of acute asthma, the treating physician performed a routine clinical assessment to evaluate the need for nebulization. Before this evaluation, the CAS was recorded by 2 independent nurses and later compared with the physician's assessment and treatment decision. For inter-rater agreement assessment, rating had to be performed successively in pairs, without intervening treatment. To ensure that enough health-care providers would be available to assess the CAS, the CAS was assessed only during office hours.
Statistics
Continuous variables were expressed as median values and interquartile range, and categorical data were presented as counts and percentages. Differences between groups for continuous data were evaluated using the Mann-Whitney U test.
Inter-rater agreement of the CAS was assessed using the weighted kappa statistic; values of 0.7 or more were considered indicative of a good level of agreement.12 Internal consistency of the CAS was assessed using the Cronbach alpha coefficient. The degree to which each individual item contributed to the overall CAS was also assessed.
The test performance of the CAS was assessed by calculating both sensitivity and specificity as well as negative and positive predictive values, with 95% exact (Clopper-Pearson) confidence limits for the binomial proportions. In addition, the receiver operating characteristic curve was estimated using logistic regression. In these calculations, all episodes of acute asthma were used as independent observations. The analyses were performed using SAS 9.2 (SAS Institute, Cary, North Carolina). P < .05 (2-tailed) was considered to be statistically significant.
Sample-Size Calculations
For this study, sample-size calculation indicated a necessary number of 50 subjects with at least 2 CAS during the course of admission for acute asthma. With 140 episodes, our study achieved at least 90% power to detect a change in sensitivity (and specificity) from 0.8 to 0.62. For this, a 2-sided binomial test was used. The target significance level was .05. The actual significance level achieved by the sensitivity (specificity) test was .36. The prevalence of the disease was 0.5.
Informed Consent
This study was approved by the local institutional review board, and all subjects received standard care for acute asthma. In accordance with Dutch legislation, written informed consent was not required in this setting.
Results
The CAS was recorded during 134 episodes of acute asthma in 47 subjects. The median number of CAS per subject was 2 (range of 1–13). Subject characteristics are presented in Table 2. Overall, in 74 episodes (55.2%), routine assessment by the treating physician led to administration of nebulization.
Subject Characteristics
CAS overall scores ranged from 2 to 10 (of a possible 12). Lower overall scores were more frequently given than higher scores (Fig. 1). Clinical assessment not leading to treatment with bronchodilators was associated with lower CAS (median of 2.5, interquartile range of 2.0–3.0) compared with that leading to bronchodilator nebulization (median of 6.0, interquartile range of 4.0–7.0, P < .001). Furthermore, for all separate components of the CAS, median scores were significantly lower for subjects who did not receive bronchodilators (Table 3).
Percentages of overall childhood asthma scores.
Median Score and Interquartile Range of Separate Components of Childhood Asthma Score for Subjects Who Did and Did Not Receive Bronchodilators After Clinical Assessment
The sensitivity and specificity of each CAS are presented in Table 4. The CAS area under the receiver operating characteristic curve is 0.937 (95% CI 0.899–0.975, P < .001) and is presented in Figure 2.
Sensitivity and Specificity for Each Childhood Asthma Score
Receiver operating characteristic curve of the childhood asthma score. Area under the curve = 0.94.
Using a CAS cutoff value of 4 or higher to administer bronchodilators, a sensitivity of 0.91 (95% CI 0.84–0.97) and a specificity of 0.77 (95% CI 0.66–0.87) were achieved. Thus, 91% of the time a subject had a CAS of 4 or higher, the treating physician had also ordered nebulization. On the other hand, 23.3% of the time nebulization was not given or was postponed until after clinical assessment, nebulization would have been initiated if based on a CAS of 4 or higher.
By using the abovementioned CAS cutoff value of 4 to initiate treatment, the positive predictive value was 0.83 (95% CI 0.75–0.91), indicating that 83% of the time a CAS was positive, there was a nebulization requirement based on clinical assessment of respiratory status. Furthermore, the negative predictive value was 0.87 (95% CI 0.78–0.96), indicating that 87% of the time a CAS was negative, nebulization was not given.
With regard to construct validity, the internal consistency of the CAS was moderate (Cronbach alpha .62); the contribution of each of the 4 components is presented in Table 5. The internal consistency did not significantly change if successive components were omitted from the overall score (data not shown).
Contribution of Each Item to the Overall Score (Cronbach Alpha)
Inter-rater agreement was assessed in 79 CAS pairs. The inter-rater agreement between the overall CAS was good (weighted kappa of 0.77, 95% CI 0.71–0.83). The agreement for individual components of the CAS was good for oxygen saturation (kappa of 0.81, 95% CI 0.70–0.91) and degree of dyspnea (kappa of 0.75, 95% CI 0.65–0.86) but moderate for breathing frequency (kappa of 0.46, 95% CI 0.22–0.71) and retractions (kappa of 0.56, 95% CI 0.43–0.70).
Discussion
Our aim was to determine the value of a newly developed asthma score in comparison with routine clinical assessment of a patient's respiratory status to predict bronchodilator nebulization in children with acute asthma. Our results demonstrate that the newly developed CAS accurately predicts the requirement for bronchodilator nebulization in children with acute asthma without use of auscultative findings. Ideally, by using the CAS only, none of the patients who would otherwise receive bronchodilator nebulization will be missed. On the other hand, however, only a small proportion will be overtreated. Indeed, the CAS correlated well with routine clinical judgment to administer bronchodilators in children with acute asthma. The high positive and negative predictive values of the CAS and the good inter-rater agreement suggest that the CAS may be an attractive tool to be implemented in a clinical pathway for acute asthma in children.
The development of a clinical pathway for children with acute asthma is worthwhile because it has been demonstrated that the implementation of structured care reduces hospital stay without increased morbidity.8,13 Although several asthma scores have been developed and proven useful in clinical pathways,1–7 all of these pediatric asthma scores require auscultation of the lungs to score the degree of wheezing or air entry. This implies that all health-care providers should be trained to auscultate the lungs. One study evaluated in-patient asthma management by means of a clinical pathway and included a weaning protocol for bronchodilator administration.13 This weaning guideline was based on 4 items, including assessment of lung sounds. In addition, a nurse determined the criteria to change the frequency of therapy and notified the house officer, who subsequently assessed the patient and ordered a change of therapy. It is common practice to adjust the frequency of bronchodilator administration by monitoring the respiratory status of the patient. Thus, in-patient care is dependent on frequent clinical assessments. By using a standardized respiratory assessment such as the CAS, the in-patient care of children with acute asthma may be improved by reducing variability in care.
Our study has limitations. The first limitation is that the study was conducted during office hours to ensure availability of nursing staff. We cannot exclude that the respiratory status of a patient is assessed differently during evening or night shifts. The respiratory status of patients may be compromised to a greater extent while sleeping due to a decrease in breathing frequency and tidal volume. Our findings cannot be extrapolated to these situations without further research.
In addition, the inter-rater agreement was good for the overall score and for the degree of dyspnea. The clinical condition of asthma varies, and this fluctuation may be the reason why the inter-rater agreement for breathing frequency and degree of retractions was lower than for dyspnea, which reflects the overall respiratory status of the patient. The fact that assessment of the CAS by different health-care providers was not performed at the same time should also be taken into account. A dichotomous scoring system such as the RAD score (respiratory rate, accessory muscle use, decreased breath sounds), as recently demonstrated, may be even easier to use with less degree of subjectivity.14 However, this score also requires auscultation of the lungs.
Furthermore, the CAS was performed only during the course of the disease and not on initial presentation at the emergency department. This may explain the small number of subjects with a high CAS. Indeed, a substantial percentage of subjects presented with mild acute asthma and did not receive systemic steroids at the discretion of the treating physician. In our practice, only patients who do not respond well to the first 2 administrations of bronchodilators receive systemic steroids. On the other hand, the CAS was developed for use in a clinical pathway and not to predict whether a patient with acute asthma should be admitted. In addition, we did not assess the correlation of CAS and response to treatment. Indeed, the feasibility of using the CAS in a clinical pathway for acute childhood asthma is currently under evaluation.
Conclusions
We developed a CAS that does not require auscultation of the lungs. The results demonstrate that the CAS correlates well with routine clinical assessment of the patient's respiratory status by a physician. This suggests that the CAS may be an attractive tool in a clinical pathway to reduce clinical variability in the management of acute childhood asthma. The next step will be to use the CAS in a prospective randomized trial to investigate whether its use improves decision making and patient outcome.
Acknowledgment
We are indebted to the physician and nursing staff of the Pediatric Department of the Medical Centre Leeuwarden for support of this research project.
Footnotes
- Correspondence: Arvid WA Kamps MD PhD, Department of Paediatrics, Medical Centre Leeuwarden, PO Box 888, 8901 BR Leeuwarden, The Netherlands. E-mail: arvidkamps{at}gmail.com.
The authors have disclosed no conflicts of interest.
- Copyright © 2014 by Daedalus Enterprises
