Abstract
BACKGROUND: Symptom burden increases during pulmonary exacerbations of cystic fibrosis (CF), and patient-reported outcome measures (PROMs) are often used to evaluate symptoms as either primary or secondary outcomes. However, there is currently no guidance on the use of PROMs to assess symptom burden during pulmonary exacerbations.
METHODS: A systematic literature search was conducted to identify PROMs measuring symptom experience, management, or influencing factors, which were developed for CF patients and had been used at least once during pulmonary exacerbations. The PROMs included were assessed for relevance and psychometrics, according to the criteria of the United States FDA guideline and the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist.
RESULTS: Five PROMs were identified, all measuring symptom perception. The CF Respiratory Symptom Diary (CFRSD) and the Symptom Scoring System were developed to assess symptom severity during pulmonary exacerbations. Of the other 3, which also included symptom scores of 2 quality of life measures, one assessed symptom severity exclusively, and 2 measured symptom severity in addition to other dimensions (such as symptom distress). All 5 instruments measured respiratory symptoms. Other relevant symptoms, such as energy and emotions, were covered by 4 instruments; pain and gastrointestinal symptoms were covered by 2 measures. All of the instruments demonstrated good internal consistency and sensitivity to change over a period up to 4 weeks. The symptom scores of the 2 quality of life measures with longer recall periods are not suitable for measuring assessed changes in a period of < 2 weeks. Criterion validity for gastrointestinal subscores has not been established. Discriminant validity was established in all of the instruments reviewed except for the Symptom Score System.
CONCLUSIONS: Of the current PROMs used during CF pulmonary exacerbations, only 2 have been developed for this purpose, and only the CFRSD fulfilled all FDA guideline criteria. To date, there is no instrument that assesses exacerbation-specific symptom distress.
- patient-reported outcomes
- questionnaire
- exacerbation
- cystic fibrosis
- symptom perception
- experience
- symptom management
Introduction
Patient-reported outcome measures (PROMs) are widely used to support clinical decision-making in patient care1,2 and to evaluate the effects of interventions.3 A PROM is defined as “any report of the status of a patient's health condition that comes directly from the patient, without interpretation of the patient's response by a clinician or anyone else.”3 PROMs play a crucial role in the assessment of symptoms, which are an individual's subjective experience.4 As experience can only be assessed by the person himself or herself, self-report by the patient is the accepted standard in symptom assessment.5 Severity, frequency, and distress are the dimensions of symptom perception that are consistently recommended to be measured by PROMs.4,6 In addition to symptom experience, symptom management and influencing contextual factors are other relevant concepts from a symptom-theory perspective.4,5
With regard to cystic fibrosis (CF) exacerbations, PROMs have been widely used to evaluate the effect of interventions on symptom severity, as either primary or secondary outcomes.7 However, severity is only one aspect of symptom perception; symptom distress may be of equal or even greater relevance for patients.8 Furthermore, symptoms9 and increased treatment burden10 can affect patients' ability to undertake self-management activities, such as adherence to treatment, which may adversely affect clinical outcomes.11–13
The aim of this literature review was to identify PROMs assessing symptom experience, management, or influencing factors in CF subjects with pulmonary exacerbations, and to provide guidance as to which PROMs are most appropriate for use during exacerbations.
We used 3 review questions:
What CF-specific measures are currently available for assessing symptom experience, symptom self-management, or influencing factors on symptom self-management during a pulmonary exacerbation?
Which concepts do these measures assess? If assessing symptom perception, which dimensions (severity, frequency, quality, or distress) of symptom experience do these measures assess?
What are the strengths and weaknesses of the measures available, specific to their use during pulmonary exacerbations?
Methods
For this review, we searched systematically for articles describing PROMs that assess symptom experience (eg, perception or evaluation), symptom management (eg, adherence) or influencing factors (eg, self-efficacy) in subjects with CF who experienced pulmonary exacerbations (Table 1).
Search Strategy
Inclusion criteria were:
Articles published in 1994 or later, written in German or English, with measures in English or German. Measures developed before 1994 were included if used in the past 20 y. Measures not used in the past 20 y were deemed to have little relevance for current clinical practice and were therefore not included.
Adult sample (a portion or all of the participants > 18 y old).
The measure had been used (at least once) during an exacerbation period.
The measure was developed for patients with CF, in either acute or stable phases.
The measure's development and validation were reported, in either stable or acute phases.
One author checked 10% of the studies deemed to be relevant, and all included studies for inclusion and exclusion criteria. Two authors conducted a quality appraisal of the included studies.
Eligible PROMs were critically appraised according to criteria developed by the United States FDA, which recommends a 5-step PROM-development strategy: 1) hypothesize the conceptual framework; 2) adjust the conceptual framework and draft the measure based on patients' input; 3) confirm the conceptual framework and assess other measurement properties; 4) collect, analyze, and interpret data; and 5) modify the measure.3 These criteria were supplemented with those of the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN), which describe a standard for high-quality methodology for studies focusing on measurement properties of PROMs. The criteria refer to reliability, content, construct, criterion validity, responsiveness, and interpretability of PROMs.14
Results
The article selection process is described in Figure 1. A total of 107 measures were initially identified. Two meta-analyses15,16 yielded adherence measures, but as their investigations did not take place during exacerbation periods, the original articles were not screened to identify further measures. Of the 107 measures, 89 had never been applied during an exacerbation. Of the remainder, 7 were applied during an exacerbation but were not initially developed for CF; for 6 further measures, no validation data were available.
Flowchart. CF = cystic fibrosis.
Only 5 measures fulfilled the selection criteria. Of these, 2 were CF-specific health-related quality-of-life measures that included symptom scores: the CF Questionnaire Revised 14+ for teens and adults (CFQ-R)17 and the CF Quality of Life (CFQOL) questionnaire.18 Three CF-specific symptom scores were also identified: the Memorial Symptom Assessment Scale for Adults with CF (MSAS CF);8 the CF Respiratory Symptom Diary (CFRSD),19 of which a short version exists, the CF Respiratory Symptom Diary–Chronic Respiratory Infection Symptom Score;20 and the Symptom Score System.21 All 5 measures selected assessed symptom perception.
We identified no CF-specific validated measures that assessed symptom self-management (self-management, adherence, and symptom evaluation or response, see Fig. 1) or influencing factors on symptom self-management (knowledge and self-efficacy, see Fig. 1) and had been applied during an exacerbation. The Self-Management Questionnaire for CF22 and the CF Self-Care Practice Instrument23 measure self-management or self-care in adults, but neither was developed for exacerbation or had been used in such episodes. Furthermore, we identified numerous measures that assessed treatment adherence in general.13,24–38 Of these, 2 studies measured adherence during an exacerbation, but they used a non-validated scale.39,40 The Perceived Health Competence Scale, which assesses self-efficacy,41 the Transitional Dyspnoea Index Score,42 the Chronic Respiratory Disease Questionnaire,43 the Pain Catastrophizing Scale,44 the Brief Pain Inventory,44 the Schwartz Fatigue Scale,45 and the Quick Inventory of Depressive Symptomatology46 were each used once during an exacerbation, but they were not developed for a CF-specific population and were therefore not included for review. Five symptom checklists had been used during an exacerbation, but no validation data were available.45,47–50
The characteristics of the 5 measures included are presented in Table 2. Their psychometrics, namely validity, reliability, responsiveness to change, and interpretability, have been assessed according to the criteria provided in the COSMIN and FDA guidelines.
Psychometric Properties of the PROMs Used During an Exacerbation to Assess Symptom Perception
All 5 selected measures were developed between 2000 and 2012. The CFQ-R and the CFQOL are CF-specific quality-of-life measures. However, their symptom scores, especially the respiratory and the digestive symptom scale of the CFQ-R, have been widely and independently used in intervention studies to evaluate treatment effects in CF. The other 3 measures are symptom scores, and while the CFQRSD and the Symptom Score System were developed for exacerbations, the MSAS-CF was intended for general use.
With regard to content, respiratory symptoms are assessed in all 5 measures, energy and burdensome emotions in 4 measures, pain and gastrointestinal symptoms in 2 measures, and fever/chill in 1 measure. An overview is provided in Table 3.
Content Covered by the Measures
The CFQ-R, the CFRSD, the Symptom Score System, and the CFQOL ask for symptoms unidimensionally in that they separately assess severity, frequency, or distress for each symptom (Table 2). Only the MSAS-CF asks for symptoms multidimensionally, indicating an assessment of all domains for each symptom. The CFQ-R, the CFRSD, and the Symptom Score System measure the severity, the frequency, or the quality of a symptom. The CFQOL assesses symptom distress, asking for “troublesomeness” and “embarrassment.” In addition to severity and frequency, the MSAS-CF asks about the distress that accompanies each symptom by asking how much “distress” or “bothersomeness” the patient associates with the symptom.
All instruments were developed on the basis of literature review, other measures, or expert opinion. Only the CFRSD, CFQ-R, and the CFQOL involved patients in the generation of content as recommended by the FDA guideline, and only the CFRSD involved patients during an exacerbation and tested items for clarity via cognitive debriefing interviews. In some instruments (eg, CFSRD), items that were bothersome to patients but had relatively low prevalence (eg, pain), were excluded.
Other than for the CFQ-R digestive score, the CFRSD emotional score, and the Symptom Score System, discriminant validity was established in all of the selected subscores. Criterion validity was established for the respiratory scores of the CFQ-R, the CFQOL (chest score), and Symptom Score System using FEV1 values. For the respiratory scores of the CFRSD and the MSAS-CF, it was established by using other self-report measures, but not FEV1 values. Emotions and energy scores of the CFQ-R, the CFQOL and the MSAS-CF were validated using other validated self-report scores as a gold standard.
For gastrointestinal-related items, no criterion validity has been established, either for the CFQ-R or for the MSAS-CF gastrointestinal scores. The MSAS-CF gastrointestinal scores correlated only weakly with the CFQ-R digestive symptom score. This is unanticipated, but could be due to the CFQ digestive symptom score, which showed an unexpected pattern in previous research.51 In addition, all instruments demonstrated good internal consistency.
All measures demonstrated sensitivity to change during pulmonary exacerbation. However, because the CFQ-R, the CFRSD, and the MSAS-CF were not developed for exacerbations, they have relatively long recall periods (ie, 1 week for the MSAS-CF and 2 weeks for the CFQOL and the CRQ-R). In testing the CFRSD's daily versus weekly recall period, the weekly scores were higher than the calculated mean score of the preceding 6 days. Significant differences were found for the mean of the five respiratory items, the 5 mood items, and the single tiredness item. These results confirm that symptom-measurement accuracy is generally higher if measured daily.60 Minimal important different scores were established on the basis of statistical analysis for the CFQ-R respiratory score and the CFRSD respiratory score.
Discussion
Five CF-specific measures that assess a symptom-specific concept and were used at least once during a pulmonary exacerbation were identified. There were 3 PROMs developed for stable phases, and 2 for exacerbations. All 5 PROMs measured symptom perception. Only the MSAS-CF, developed for stable phases, assessed severity, frequency, and distress for each symptom. The other instruments asked for severity, frequency, quality, or distress for 1 symptom. Of the 2 exacerbation-specific PROMs, the CFRSD assessed symptom severity exclusively, while the Symptom Score System assessed severity, timing, or quality of 1 symptom. With regard to content validity, all 5 instruments measured respiratory symptoms. Other relevant symptoms, such as energy and emotions, were covered by 4 instruments, with pain and gastrointestinal symptoms covered by 2 measures. All instruments demonstrated good internal consistency and sensitivity to change for up to 4 weeks. However, the symptom scores of the 2 QOL measures have a recall period of 2 weeks, making them unsuitable for measuring change during exacerbation periods of < 2 weeks. Criterion validity has been established for most respiratory, emotional, and energy scores, but not for gastrointestinal scores.
One critical issue is that CF subjects were only involved in the development of the CFQ, the CFQOL, and the CFRSD, and only the CFRSD involved subjects in testing the clarity of its items. The lack of patient involvement in developing certain measures gives rise to the question of whether the content validity of those instruments is actually given. Only 1 measure's tool development (CFRSD) included interviews with subjects experiencing a pulmonary exacerbation. However, subjects were interviewed only at the start of the exacerbation, coinciding with the time when all prevalence-based decisions were being made regarding the inclusion of items. This may be a critical issue regarding the instrument's content validity, as certain symptoms may develop in response to other symptoms or to treatment during an exacerbation. For example, pain due to coughing, lack of muscle strength due to lack of physical activity, and weight loss due to lack of appetite have been reported as relevant symptoms during pulmonary exacerbations from the patient's perspective.9,61 The role of gastrointestinal symptoms with regard to exacerbations must also be clarified. Although diarrhea and nausea are frequently reported adverse effects of antibiotic treatment, which is a common treatment of pulmonary exacerbation in CF, their relevance for patients in the course of the exacerbation is not clear at this time. To explore the evolution of symptoms over time and to minimize recall bias, a recurring qualitative measure design with several interview time points should be applied in future research.62
The CFRSD, CFQ-R, and the Symptom Score System measure symptom severity. Similar to the CFRSD, the Exacerbations of Chronic Pulmonary Disease Tool, an instrument measuring the effect of treatment on exacerbation in COPD, assesses symptom severity and frequency but not symptom distress.63,64 PROMs that assess symptom severity and frequency may be suitable for detecting and assessing the severity of an exacerbation in CF and COPD. However, the limitations of these symptom dimensions (severity and frequency) may make these instruments less suitable for guiding and evaluating patients' symptom management. Given that symptom distress is a driver in patient self-management, inclusion of the distress dimension in symptom assessment is essential in the planning and evaluating of interventions, especially in terms of self-management.4 This is especially relevant in CF, as earlier research emphasizes that CF subjects perceive symptom severity and distress as different dimensions.8 Still, it remains unclear whether symptom severity truly reflects a dimension of importance for patients. This must be discussed critically in future research. Regardless, future developers of PROMs in CF will have to consider how to include the dimension of symptom distress in instrument development, along with how “distress” should be measured and how to formulate appropriately inclusive wording. To date, the 2 CF PROMs that assess symptom distress use different wording: where the CFQOL assesses distress as “troublesomeness” and “embarrassment,” the MSAS-CF assesses “distress” or “bothersomeness.” The lack of standardized wording for symptom distress in CF PROMs may distort the comparability of symptom distress between the different measures. Research from other populations with long-term conditions indicates that subjects experience symptoms that interfere with normality and daily life as burdensome, which could be a dimension of symptom distress for CF patients as well.65 The matter of which symptom characteristics lead to distressing exacerbation experiences remains to be explored.
Of the instruments in question, only the CFRSD covered all aspects of the FDA guidance and was developed specifically for use in exacerbations. It is currently held to be the most appropriate instrument for assessing symptom perception during a pulmonary exacerbation. However, a critical issue in the development of the CFRSD is that items were derived from patient narratives at the beginning of the exacerbations. There is a lack of knowledge about how patients experience symptoms during exacerbation. This means that there is currently no definitive answer as to whether the CFRSD incorporates all relevant symptoms over the course of the exacerbation. One limitation is that it assesses only symptom severity, although symptom distress may be a further relevant dimension.
A limitation of this review is that the content that should be covered by a PROM was not able to be defined from the patient perspective. This is due to the current lack of qualitative data regarding CF patients' experience of a pulmonary exacerbation. In the review, we addressed this limitation by appraising whether a PROM development involved patients.
Future research must explore patients' experience of an exacerbation, preferably using a longitudinal design. Additionally, symptom distress in CF requires further conceptualization. This knowledge will provide a basis for the development of a PROM to assess symptom distress during pulmonary exacerbation.
Conclusion
Of the 5 PROMs that were included in this review, only the CFRSD fulfilled all the criteria of the FDA PROM development guidelines and was developed for pulmonary exacerbations. However, because items for the PROM were derived from patient data at the beginning of exacerbations, this may be a critical issue for content validity in the CFRSD. A limitation of the CFRSD is that it assesses only symptom severity. Because evidence indicates that symptom distress is a relevant concept from the patient's point of view, PROMs including such a dimension are needed. Further research should explore patients' experience of pulmonary exacerbation and provide a basis for conceptualizing the symptom distress associated with pulmonary exacerbation.
Acknowledgments
We thank Chris Shultis de Windter and Jane White for editing assistance.
Footnotes
- Correspondence: Gabriela Schmid-Mohler MSN RN, Centre of Clinical Nursing Science, University Hospital Zurich, Rämistrasse 100, CH-8091 Zürich, Switzerland. E-mail: gabriela.schmid{at}usz.ch
The authors have disclosed a relationship with the Nursing Science Foundation Switzerland.
- Copyright © 2018 by Daedalus Enterprises
