Abstract
BACKGROUND: In 2008 the Cystic Fibrosis (CF) Foundation launched the Respiratory Therapy Mentoring Program, which pairs a respiratory therapist (RT) relatively new to CF (apprentice) with a highly experienced RT (mentor) from a similar CF care center. We wished to determine if we had achieved our short-term goal of increasing CF-specific knowledge among the apprentices who participated in the program.
METHODS: Selected apprentices were each matched with a mentor, based on characteristics of CF population, clinical setting, center size, and geographic location of their care centers. Apprentices completed a CF-specific RT knowledge self assessment tool prior to and after a site visit to their mentor's center. Mentors also completed a post site visit knowledge self assessment tool regarding their apprentice.
RESULTS: Thirty-seven apprentices completed a pre and post site visit knowledge self assessment tool. The median pre and post site visit scores were 12 and 31 (P < .001) respectively. The mentors' post site visit scores of their apprentices (median 29, P = .07) did not significantly differ from the apprentices' post site visit scores.
CONCLUSIONS: The results of this preliminary evaluation suggest that the RT mentoring program has achieved its short-term goal of increasing CF-specific knowledge among RTs relatively new to CF care.
Introduction
Cystic fibrosis (CF) is an autosomal recessive, multi-system chronic disease affecting an estimated 30,000 people in the United States. The pulmonary manifestations account for the majority of the morbidity and mortality.1 Patients with CF have difficulty clearing mucus from their lungs, resulting in chronic pulmonary infections and inflammation, which leads to bronchiectasis. In order to maintain lung health, clearance of airway secretions is recommended.2 Multidisciplinary care, aggressive pulmonary and nutritional management, and newly introduced therapies have resulted in significant improvements in the length and quality of life for people with CF.
The CF Foundation accredits more than 110 CF care centers across the United States, caring for approximately 25,000 patients. The care centers are comprised of multidisciplinary teams that include physicians (typically pulmonologists serving as center directors), nurses (typically serving as clinical coordinators), respiratory therapists (RTs), dietitians, social workers, and other healthcare professionals.3 The CF RT is an integral member of the team. The optimal care of a CF patient requires knowledge of the disease and its impact on the lungs and other organ systems, pulmonary function testing, airway clearance techniques, chronic medications for maintenance of lung health, pulmonary complications, and infection control issues.2,4–6 Practicing RTs must also build a strong partnership with patients and families in order to customize care to the needs and preferences of the individual and promote self-management strategies and adherence to the daily medical regimen. Because of the specialized nature of CF care, CF RTs in the United States gain most of their knowledge and experience through on-the-job training from other members of their care center team and annual participation in courses and workshops at the North American Cystic Fibrosis Conference.
As part of the CF Foundation's ongoing quality improvement initiative, the Foundation launched the CF Respiratory Therapy Mentoring Program in 2008. The primary goal of this study was to assess the effectiveness of this new program by measuring CF-specific knowledge among apprentices with a CF-specific RT Knowledge Self Assessment Tool (RT KAT) designed for this study, before and after completion of the program. In addition, we conducted surveys of the apprentices and their center directors/clinic coordinators to collect subjective feedback on the program.
QUICK LOOK
Current knowledge
The Cystic Fibrosis (CF) Foundation accredits more than 110 CF care centers across the United States, which care for approximately 25,000 CF patients. The CF respiratory therapist (RT) is an integral member of the team. Optimal care of a CF patient requires knowledge of the pathophysiology, pulmonary function testing, airway clearance techniques, lung health maintenance medications, pulmonary complications, and infection control issues. In the CF Foundation's respiratory therapy mentoring program, experienced mentors give RTs specialized training in CF respiratory care.
What this paper contributes to our knowledge
The CF Foundation mentoring program was found to effectively improve apprentice RTs' knowledge of this disease.
Methods
The CF Foundation, which provided administrative and financial support for the mentoring program, chose 3 highly experienced RTs to serve as facilitators for the program and to develop a CF Respiratory Therapy Resource Document (RT 101). This comprehensive document covered the fundamental knowledge thought to be important in delivering competent respiratory care to CF patients. It is available to all RTs on the CF Foundation's intranet.
RT Knowledge Assessment Tool
The RT 101 document served as the template for development of the RT KAT, an assessment tool that encompassed a wide range of topics pertinent to CF patient care, such as airway clearance techniques, infection control, equipment and devices, and inhalation therapy. It consisted of 13 questions, with 19 scored responses (see the supplementary materials at http://www.rcjournal.com). The selected mentors reviewed the RT 101 document and the RT KAT before launch of the mentoring program, and there was a consensus among these experts that the RT KAT covered the key areas of CF-specific knowledge for RTs working at a CF care center (“face validity”). No formal validation or psychometric analyses were conducted on the RT KAT.
Apprentices were asked to provide an honest self-assessment of their RT knowledge and skills at both the beginning and end of the program (pre and post site visit RT KAT). Each mentor was also asked to complete a post site visit RT KAT to assess their apprentice's knowledge. All the answers were scored using the following Likert rating scale: Strongly disagree (−2), Disagree (−1), Agree (1), Strongly agree (2). The minimum and maximum scores were –38 and +38, respectively.
Program Evaluation Surveys
To further assess the effectiveness of the mentoring program, 2 surveys were conducted. First, a survey of the apprentice's center director and/or clinic coordinator was conducted 3–6 months after the site visit. This survey evaluated the apprentice's impact on their center's CF clinical practice.
Second, in the fall of 2010, a survey was conducted of all former apprentices. The completion of this survey was anonymous, preventing the identification of the responders and non-responders. It assessed the apprentice's confidence in performing their role and perceived improvement in CF patient care at their center. Questions regarding continued engagement in CF care and amount of time spent caring for CF patients were also asked to assess the retention of RTs after participation in the program. Both surveys were conducted using Survey Monkey (www.surveymonkey.com).
Selection of Participants and Matching
Starting with a successful pilot, from 2008 to 2010, the program consisted of 2 sessions per year, with approximately 10 apprentice/mentor matches per session. Of the 81 apprentices who applied, 44 were chosen to participate, based on criteria described below. In 2011 the program was changed to 1 session per year, with 10–15 matches.
A call for applications was sent out at the beginning of each session, via the CF RT Listserv and an e-mail communication to all CF care center directors. A program overview was provided, which outlined the roles and expectations of the mentors and apprentices. All applications were assessed objectively using a point system.
Mentor applications were evaluated and scored based on years of CF experience (a minimum of 5 years is required), number of hours worked, percent of time devoted to CF care, and level of involvement in the CF respiratory therapy community (eg, participation at national conferences, consortia). Selected mentors agreed to participate in the program for a minimum of 2 sessions.
Apprentice applications were evaluated and scored based on years of CF experience (< 3 years was prioritized), center size, number of hours worked, percent of time devoted to CF care, and the availability of other CF RTs at the center. For example, applicants with less CF experience, working at a high volume CF center and with no other experienced CF RTs, scored higher than those with more CF experience at a smaller center with the support of other CF RTs. Those with the highest point totals were selected to participate in the program.
The mentors and apprentices were then matched based on the characteristics of the CF population they served (adults/pediatrics/both), their clinical setting (in-patient/out-patient/both), center size, and the geographic location of their respective CF centers.
Site Visit Process
Mentors and apprentices were notified of their acceptance into the program and were provided contact information for their counterpart, a checklist of responsibilities, guidelines for developing learning objectives, and a copy of the CF RT 101 resource document. Apprentices were also asked to complete the pre-site-visit RT KAT. The completed RT KAT and the apprentice's application were provided to his/her mentor. Conference calls were then conducted between the facilitator and each mentor/apprentice pair to review expectations and site visit logistics.
Each apprentice developed learning objectives for the site visit with input from his/her own CF care team and the assigned mentor. Once the objectives were finalized, the mentor prepared an agenda for the 1–2 day site visit. The visit typically included a meeting with the multidisciplinary CF team, a review of the RT 101 document and the mentor's CF respiratory specific educational and patient assessment materials, shadowing the mentor in out-patient clinic and/or during in-patient rounds, pre and post clinic conferences, and a debrief at the end of the visit to review the learning objectives. Within 2 weeks after the site visit, the apprentice and mentor completed a post site visit RT KAT.
Outcome Measures
The RT KAT score of the apprentices and mentors was the primary outcome measure. Apprentices who completed both a pre and post site visit RT KAT were included in the analysis. If an apprentice did not answer a specific question on either the pre or post site visit RT KAT, that question was excluded from the analyses. If an apprentice or mentor did not answer a specific question on the post site visit RT KAT, that question was excluded from the analyses. The total scores are presented as medians with minimum and maximum values.
Statistical Analyses
The pre and post apprentice scores for each question and for the sum totals were compared with a Wilcoxon signed-rank test. The apprentices' and mentors' post site visit RT KAT scores for each question and the sum totals were compared with a Mann-Whitney test. A P value (2 tailed) < .05 was considered statistically significant.
Results
Participants
Since inception, 44 apprentices and 23 mentors have participated in the program. The apprentices were selected from among 81 applicants.
As shown in Table 1, the mentors had a median of 20 years of experience in CF care (range 7–30 y). The mentors were from both pediatric and adult CF care programs and various center sizes, with a median of 170 patients (range 90–600 patients). The median proportion of time that the mentors devoted to CF care was 55% (range 25–100%). Twenty of the 23 mentors worked in both the out-patient and in-patient setting, and only 3 of the 23 provided care exclusively in the out-patient setting.
Characteristics of Apprentices and Mentors: 2008–2010
Of the apprentices who participated in the program, 33 of 44 had < 3 years experience in CF care, and 20 had < 1 year. The apprentices also represented pediatric and adult programs, with 23 of 44 working in both. The median center size among apprentices was 120 patients (range 40–400) and the median time devoted to CF was 38% (range 15–100%). Twenty-nine of the 44 apprentices provided care in both the in-patient and out-patient setting.
RT KAT Results
Thirty-seven mentor/apprentice pairs (excluding the pilot session) completed a pre and post-site visit RT KAT. Table 2 shows the distribution of apprentice responses to each question in the pre and post site visit RT KAT. Of note, before the site visit, over half of the apprentices answered “disagree” when asked if they were able to discuss, demonstrate, and instruct patients using airway clearance techniques, such as active cycle of breathing, autogenic drainage, and intrapulmonary percussive ventilation, and over a third disagreed with questions concerning the their own knowledge of transmission of pathogens, medication use at their care center, adverse side effects of medications, and educational resources for patients and families.
Distribution of Apprentices' Knowledge Self Assessment Responses
The median apprentice pre site visit total score was 12 (range –13 to 35) and the median apprentice post site visit total score was 31 (range 17 to 38), a statistically significant difference (P < .001). The difference between pre and post site visit scores was also statistically significant for each individual question on the RT KAT (see Table 2). The apprentices' and mentors' post site visit RT KAT scores did not significantly differ: median of 31 (range 17 to 38) and 29 (range 12 to 38), respectively (P = .07). The apprentices' and mentors' post site visit scores also did not differ significantly for any of the individual questions (data not shown).
Survey Results
Twenty-six of 37 (70%) of the apprentices' center directors/clinic coordinators completed the survey, and 72% responded that CF respiratory care had improved at their center since the RT participated in the program, and 80% responded that their RT was more knowledgeable and active in CF care. Among the 44 apprentices, 38 (86%) completed the program evaluation survey. The results are shown in Table 3. Of note, 33 of 38 (86.8%) reported that the level of respiratory care had improved at their center since participation in the program.
Apprentice Program Evaluation Survey
Discussion
The CF Foundation mentoring program provided an opportunity for an RT relatively new to the role as a member of the CF multidisciplinary care team to work closely with and learn from a highly experienced RT. In this study we found that apprentices' self-assessed CF-specific knowledge significantly increased following participation in the mentoring program. Results of the program evaluation surveys completed by the apprentices and their center directors/clinic coordinators provide supportive evidence of the value of the program. These findings suggest that the mentoring program is meeting the short-term goal of increasing CF-specific knowledge among RTs who are relatively new to CF care.
Anecdotal evidence pointed to additional unexpected benefits of the program. First, we believe the high retention rate reported among apprentices is an important finding that may be related to improved job satisfaction. An engaged and experienced workforce will increase the likelihood that we will achieve our overarching goal (ie, to improve the quality of respiratory care and healthcare outcomes for CF patients). Of note, several apprentices have now returned to the program as mentors, including one who is currently serving as a facilitator to oversee the entire program. Second, subjective feedback from mentors suggested that the program benefitted them as well. Some mentors reported that they increased their own knowledge and confidence in preparing for their apprentice. As Aristotle once said, “Teaching is the highest form of understanding.” Lastly, the program seemed to stimulate productive interactions between RTs at care centers across the country, fostering what Wenger and Snyder7 have referred to as “communities of practice.” This we believe will be important in sustaining and further improving the CF care model.
We are not aware of any similar mentoring programs for RTs in this or any other specialized care areas. The annual North American CF Conference has sessions tailored to RTs, including a day-long session focused on teaching airway clearance techniques. This session is popular among RTs who are new to CF care and is rated highly in post-session surveys. However, this session focuses on only one aspect of respiratory care. There are other networking sessions for RTs at the North American CF Conference for informal sharing of knowledge, as well as an active Listserv, on which CF RTs can post questions and comments. None of these opportunities have one-on-one interactions focused on improving knowledge and skills, like the mentoring program described in this study.
While this preliminary evaluation of the effectiveness of the program is encouraging, several important limitations must be pointed out. The RT KAT was developed by highly experienced RTs and had face validity among their peers, but it is subjective in nature and not scientifically validated. Some key aspects of CF respiratory care may have been missed and/or others wrongly weighted in the scoring scheme. Another potential limitation is the risk of bias in the apprentice self assessments. However, the close agreement between the apprentice and mentor post site visit RT KATs lends credibility to the self assessments of the apprentices.
It is also important to note that the mentoring process was not rigorously standardized. For example, there were no processes to ensure that all mentors administered and taught airway clearance techniques in the same manner. The variability in knowledge, skills, and teaching ability of the mentors could impact the fidelity of the knowledge transmitted to the apprentice, and the subsequent assessments of knowledge by apprentice and mentor.
The program evaluation surveys also had limitations. They were subjective in nature, and the responses to the apprentice survey were collected from 1 month to 28 months post site visit. Because of the variable time elapsed between the mentoring session and completion of the survey, the results may be influenced by recall bias.8 Nonetheless, the apprentice survey results (see Table 3) showed that many of them maintained contact with their mentors and felt comfortable contacting them with questions regarding the care of CF patients after completion of the program.
Conclusions
The CF Foundation Respiratory Therapy Mentoring Program is a unique opportunity for RTs relatively new to CF (apprentices) to receive specialized training in CF respiratory care from a highly experienced mentor. This preliminary assessment suggests that the program has been effective in meeting the short-term goal of improving apprentices' RT specific knowledge.
Acknowledgments
We thank Michael Mundorff MBA MHSA and Jeanette Seidelman RRT for their help in editing this paper; Karen Homa PhD for advising us on the analytical plan and conducting the analysis; Amanda Leonard MPH RD CDE and Teresa Schindler MSc RD, the facilitators of the dietitian's mentoring program (it is the insightfulness, enthusiasm, and passion of Amanda that was the inspiration for a mentoring program); and the facilitators' CF Center directors, Barbara Chatfield MD and Patrick Flume MD, for their unending support of our endeavors in the pursuit of improved CF respiratory care.
Footnotes
- Correspondence: Kathleen M Richards RRT, Pulmonary Function Laboratory, Primary Children's Medical Center, 100 North Mario Capecchi Drive, Salt Lake City UT 84113. E-mail: kathleen.richards2{at}imail.org.
Supplementary material related to this paper is available at http://www.rcjournal.com.
The authors presented a version of this paper at the North American Cystic Fibrosis Conference, held November 3–5, 2011, in Anaheim, California.
Ms Richards has disclosed a relationship with Pharmaxis. Ms Lester has disclosed a relationship with the Cayston Allied Health Care. The other authors have disclosed no conflicts of interest.
See the Related Editorial on Page 889
- Copyright © 2013 by Daedalus Enterprises
