Abstract
BACKGROUND: New graduate respiratory therapists (RTs), regardless of the degree program, receive limited preparation in neonatal/pediatric diseases and management. Experienced RTs typically have adult knowledge but limited exposure to pediatrics. We developed a program that included competence-based simulation to improve orientation success.
METHODS: A 9-week orientation program curriculum with simulation-based competence assessment was developed to ensure all new hires gained knowledge and skills to perform pediatric clinical tasks. Each new hire individually completed the same simulation scenarios during the first week and last week of orientation. Curriculum changes were made over time based on performance in simulations and on-the-job knowledge and skills during and after orientation. Paired and unpaired t tests were used with P < .05 as significant.
RESULTS: From January 2017–February 2020, the program had 3 updates. Noninvasive ventilation and decompensating patient scenarios were completed for all periods. Ninety-two new staff were oriented in period 1 = 29 (new graduate RTs 20, experienced RTs 9); period 2 = 17 (new graduate RTs 10, experienced RTs 7); period 3 = 24 (new graduate RTs 21, experienced RTs 3), and period 4 = 22 (new graduate RTs = 22). Remediation during orientation occurred in 15% of the staff. Seventy-one percent successfully advanced to ICU orientation after completion of the program. All staff improved scores between pre- versus post-simulations in all periods: mean difference ± SD period 1: new graduate RTs 32.0 ± 17.0, P < .001; experienced RTs 28.0 ± 18.9, P < .001; period 2: new graduate RTs 23.0 ± 15.2, P < .001; experienced RTs 29.0 ± 12.1, P < .001; period 3: new graduate RTs 26.0 ± 15.8, P < .001; experienced RTs 27.0 ± 15.1, P = .007; and period 4: new graduate RTs 19.0 ± 14.5, P < .001, paired t test. The scores between new graduate RTs and experienced RTs during post-simulation were not significantly different for period 1 (P = .35) but were significantly different for periods 2–4 (P = .040, unpaired t test).
CONCLUSIONS: The use of a competence-based orientation program showed educational advancements and helped determine successful orientation completion.
Introduction
The orientation and competency assurance of respiratory therapists (RTs) are fundamental to safe and effective care of the neonatal/pediatric population. Traditionally, the standards for orientation and competency are established by The Joint Commission1; however, the orientation time frame and specific competencies are determined by each hospital to guide the RT’s adjustment to the complexity of the unique environment and various essential job functions.1 Furthermore, competence can be described as a combination of specific observable and measurable knowledge, skills, and attitudes from basic to advanced that constitute an RT’s clinical performance.2
New graduate RTs, regardless of the degree program, receive limited education in neonatal and pediatric diseases and management. Experienced RTs typically have basic respiratory care knowledge but limited clinical and didactic exposure to pediatrics. This knowledge gap leads to a steep learning curve for RTs to meet the basic competence needed to deliver safe and effective neonatal and pediatric respiratory care. The care of the neonatal/pediatric patient requires in-depth knowledge of anatomy and physiology, pathogenesis, pharmacology of the diseases, and diagnostic and therapeutic equipment and procedures.3 All accredited respiratory care programs must graduate RTs with the designated 67 entry-level competencies set by the American Association for Respiratory Care and the Commission on Accreditation for Respiratory Care (CoARC) standards.4 In addition, respiratory care programs are required to provide basic teaching on neonatal and pediatric respiratory care. CoARC does not specify the percentage of the curriculum that should be devoted to this topic and allows programs and institutions to develop those in a manner that best suits their programs.4
Simulation has been used in health care to educate health care providers and improve patient outcomes.5,6 Commonly, neonatal and pediatric training has used a more on-the-job training model for orientation, but nontraditional methods (eg, high-fidelity simulation) are being used as ways to better identify and close the knowledge and skill gaps, tailor orientation to each learner’s needs, and accelerate the learning process to obtain minimal competence at the end of orientation.7 The objective of this manuscript is to describe a competence-based simulation program built upon the new RT academic foundational knowledge to help improve successful completion of orientation. We hypothesized that the simulation score for competency assessment would increase between before and after orientation and may guide decisions regarding readiness for orientation completion.
QUICK LOOK
Current Knowledge
Onboarding of new respiratory therapist (RT) orientee is a process that every respiratory department has in place. There is no standardized orientation process that has been developed to fit every department. The orientation process varies significantly from institution to institution.
What This Paper Contributes to Our Knowledge
Our department invested in developing an orientation program that addressed many common issues within our hospital among experienced and new graduate RTs. Implementing a formal orientation program with competence-based simulation helped us address the unique issues and needs within our respiratory care department.
Methods
We developed a hospital-based orientation program to ensure all new hire RTs gain foundational knowledge and skills to perform pediatric clinical tasks. The orientation curriculum included general care clinical training with a preceptor, classes for the new hire, hands-on equipment training, skills workshops, before and after orientation competence-based assessment simulations, and orientation competencies. The orientation flow diagram is depicted in Figure 1. We used the program for all new hires, regardless of work experience. We continuously updated the content of the curricula based on new employee performance in clinical and didactic learning opportunities (Supplemental Material A, see related supplementary materials at http://www.rcjournal.com).
Orientation flow diagram from on-boarding to completion.
The orientation program included general care clinical training to focus on pediatric skills. Experienced RTs received 6 weeks and new graduate RTs received 10 weeks of orientation. The additional 4 weeks for the new graduate RTs allowed for extra clinical exposure and more time with a preceptor to focus on time management and critical thinking skills. After completion of general care orientation, experienced RTs progressed to intensive care orientation. The new graduate RTs continued to work in the regular in-patient floors and pulmonary floor for at least 3 months before advancing to intensive care orientation.
Didactic learning included classes for the new hire, hands-on equipment training, and skills workshops. The classes were held in person, and topics included computer training, asthma and bronchiolitis pathways, and the tobacco cessation program. In-person hands-on skills workshops included tracheostomy tube changes, emergency algorithms, noninvasive ventilation devices, and noninvasive interfaces. In-person hands-on equipment training covered a general overview of oxygen delivery devices, respiratory pharmacology, medication delivery devices, family/caretaker education documentation, airway clearance devices, and manual ventilation devices.
We utilized competence-based assessment simulations to evaluate new hire progress throughout orientation. The pre-orientation simulations were conducted within the first 2 weeks of general care orientation, and post-orientation simulations occurred before the last week of orientation. Scenario topics included a noninvasive ventilation patient, a decompensating patient, and a tracheostomy patient. These topics were selected because new employees in our department have found these patient care experiences to be the first stressful situations encountered in the clinical environment. Nine pediatric core scenarios were developed based on real patient care cases. Experienced RTs participated in all 3 simulations for pre-orientation simulations. New graduate RTs did not participate in the tracheostomy patient simulation due to their limited knowledge of tracheostomy change procedures. The post-orientation simulation scenarios were identical to the pre-orientation simulations, so the progression of orientation could be evaluated. Each RT participated in 3 individual scenarios. Both experienced and new graduate RTs completed all scenarios for post-orientation simulations.
We conducted the simulations on site in the Center for Simulation. An example scenario is shown in Supplemental Material B (see related supplementary materials at http://www.rcjournal.com). A Pediatric HAL (Gaumard Scientific, Miami, Florida) 5 y old patient simulator and vital sign monitor were used. The monitor display mimicked a bedside hospital monitor. The simulator provided physical cues including breath sounds, speech, chest rise, and breathing frequency. Additional clinical information was provided to the participants due to the inability of the simulator to show work of breathing. A hospital-equipped noninvasive ventilator with an appropriate interface was used for the noninvasive scenario. Oxygen delivery devices or noninvasive ventilators were used for the decompensating patient scenarios. A 4.0 uncuffed pediatric Shiley tracheostomy tube (Medtronic, Mansfield, Massachusetts) with a foam tracheostomy tube holder (Dale Medical Products, Franklin, Massachusetts) was used for the tracheostomy scenario.
Five respiratory educators, all trained in simulation-based education, facilitated the scenarios throughout all time periods. Three educators were present for each simulation session; one facilitated, one calculated the scores, and one operated the simulator. The facilitator reviewed the program, simulator, and simulation environment. A verbal handoff was given to provide scenario information. The simulator operator adjusted the vital signs as needed throughout the scenario. An abbreviated debriefing was provided after the pre-orientation assessment simulation since the post-orientation simulation topics were identical. A full debriefing was completed after the post-orientation simulations.
We developed a simulation-based assessment checklist using existing departmental procedure documents for each simulation scenario (Supplemental Material C, see related supplementary materials at http://www.rcjournal.com). Each scenario checklist had corresponding operational definitions for full completion or partial completion of technical and nontechnical skills (Supplemental Material D, see related supplementary materials at http://www.rcjournal.com). Two points were awarded if a participant performed the expected action on the checklist, one point for partial completion, and no points if the item was not done or done incorrectly. Each checklist had a total number of possible points. The educator kept track of the points accrued throughout the scenario, and participants were given a percentage score based on their performance.
We performed competency validation to determine the new hire’s knowledge and orientation progression. Leadership members in the respiratory department selected orientation competency topics for all new hires that were divided into 2 separate competency validation sessions (Supplemental Material E, see related supplementary materials at http://www.rcjournal.com). The first competency validation session included basic respiratory care topics, whereas the second session included more advanced respiratory care topics. These competency topics were selected based on The Joint Commission’s orientation competencies to provide initial training and information while assessing the competence of clinical staff relative to job responsibilities and the organization’s mission and goals.1 Competency validation sessions were based on departmental procedures and were administered by clinical educators. A 5-point Likert quality of performance rating scale was used, ranging from unacceptable to outstanding.
We used post-orientation simulation scores to guide the outcome of orientation by providing feedback and remediating key topics. Participants of post-orientation simulations were required to obtain a passing simulation score > 70%. We provided competency remediation when minor or significant errors were noted and prompting or interventions were required. Those who did not meet the passing criteria for simulations and competencies were assigned additional orientation time or instruction.
The timeline of the entire program with dates and descriptions is shown in Figure 2. Period 1 was the initial program. Period 2 included the introduction of the weighted checklist. Period 3 included an introduction to simulation video for new hires to view before the simulation experience to help decrease anxiety. Period 4 included the expansion and modification of didactic classes, the addition of a simulation workshop, and the introduction of clinical proficiency observations. We made these modifications based on participant performance and anxiety over the simulation assessment process. We updated the simulation checklist to include weighted scoring (selected item was weighted twice more) to emphasize most critical actions during the simulation scenario.
Description of changes made to the program in each time period with dates associated with each time period. NIV = noninvasive ventilation.
We implemented 7 weekly didactic classes in period 4 to improve competency achievement and simulation performance. Topics included physical assessment, disease processes, medication administration, oxygen therapy, asthma and bronchiolitis, secretion clearance, tracheostomies, noninvasive ventilation, noninvasive monitoring, and advanced assessment. The classes were recorded to be accessible on demand and repeated as often as required. To better prepare participants for the simulation, we developed a group workshop to cover the topics of the nasopharyngeal airway, escalation of oxygen to high-flow nasal cannula, escalation of noninvasive ventilation, and manual ventilation. We also introduced clinical proficiency observations. A clinical specialist, supervisor, or educator observed the new hire in the clinical environment using a clinical workflow outline. This observation helped determine the new hire’s ability to complete orientation.
Statistical Analyses
We performed a retrospective review of the performance of new hires from February 2017–November 2019 that progressed through the new orientation program. We compared simulation-based assessment scores between each time period and between new graduate RTs and experienced RTs. Categorical data are presented as n (%) and continuous data as means ± SD. Data were analyzed using SPSS 25.0 (IBM, Armonk, New York). Paired and unpaired t tests were performed for continuous data. P < .05 was considered statistically significant. The local institutional review board granted this study an exemption.
Results
Ninety-two new hire orientee records were analyzed for the study period. The full breakdown of demographics by time period is shown in Table 1. In brief, 73 were new graduate RTs, and 19 had prior clinical experience of some kind. A summary of highest degree earned before hire at The Children’s Hospital of Philadelphia showed 48 with an associate degree, 42 with a baccalaureate degree, and 2 with a graduate degree. At the time the study data were analyzed (May 2021), 15% of orientees required remediation during orientation, 71% progressed to ICU orientation, 28% resigned or were terminated before starting ICU orientation, and 1% remained employed without progressing to ICU orientation, as they were not achieving the minimum number of work hours to start ICU orientation. The orientees who resigned left the hospital for personal reasons unrelated to orientation, and some orientees were terminated due to unsafe patient care in the clinical environment after additional orientation time, instruction, and remediation.
New Hire Orientee Demographics by Time Period
Of the 92 orientees, 87 fully completed orientation and had pre- and post-simulation scores to compare for this analysis. The simulation scores for the noninvasive ventilation and decompensation simulation scenarios were used for this analysis because both new graduate RTs and experienced RTs completed these simulations. A comparison of the combined pre-orientation and post-orientation simulation scores, broken down by time period and experience level, is shown in Table 2. Both new graduate RTs and experienced RTs showed significant improvement in their simulation scores after orientation was completed (P = .007). The post-simulation scores between new graduate RTs and experienced RTs did not differ in the baseline program in time period 1 (P = .35) (Table 3). The post-simulation scores were significantly better for experienced RTs compared to new graduate RTs during time periods 2–4 (P = .046).
Comparison of Before and After Orientation Simulation Scores by Experience Level
Comparison of New Graduate Versus Experienced Post-Orientation Simulation Scores
Discussion
Our goal was to develop a hospital-based orientation program for RTs to ensure all new hires gain the foundational knowledge and skills to perform pediatric clinical tasks. We created a competence-based simulation assessment to identify and close the knowledge and skill gaps, tailor orientation to each learner’s needs, and accelerate the learning process to obtain minimal competence at the end of orientation. The increase in simulation scores from pre to post, whether in the baseline or updated curriculum, indicated the orientation program was beneficial to our new employees. The use of simulation and competency-based assessments allowed us to identify previously unidentified knowledge and skill gaps in learners that gave us the ability to tailor orientation to meet the needs of the learners and guided the decisions regarding new employee readiness to complete orientation. The entire orientation program contributed to overall new employee success, with only 15% requiring remediation.
Whereas an overwhelming majority of RT education is dedicated to adult clinical care, the care of the neonatal/pediatric patient requires in-depth knowledge of anatomy and physiology, pathogenesis, pharmacology of the diseases, and diagnostic and therapeutic equipment and procedures.3 A similar study of newly hired pediatric ICU nurses identified that orientation requires a comprehensive program to help new nurses acquire clinical skills and critical thinking due to the complexity of the pediatric patient population.6 Simulation-based assessment may also provide a method to measure clinical knowledge and skill acquisition as well as the development of competency during orientation.6
Other health care professions have used simulation-based assessment for evaluative purposes. The evolution toward the use of simulation as a mechanism for evaluating and assessing the clinical competency of individual clinicians has been seen in recent years.7 Clinical simulation is also an educational approach in many Canadian respiratory therapy programs, being strongly associated with improved learning, clinical and non–clinical skill, and future performance.8 In a study with pediatric rapid-response teams, the management of any particular scenario was reasonably predictive of overall team performance, and simulation also provided a method to develop decision making skills in managing decompensating pediatric patients.9 Simulation has also been used with extracorporeal membrane oxygenation (ECMO) specialists to determine whether scenarios can provide reliable and valid measures of a specialist’s advanced skill in managing ECMO emergencies.10 Participants endorsed these sessions as important and representative of events that might be encountered in practice.10 It is anticipated that health care professionals can expect to see increasing use of simulation for evaluation and assessment of clinical competencies.7
Simulation-based assessment provided each participant with the opportunity to learn and demonstrate skills in an environment that replicated a patient care situation they may not have readily encountered during orientation. In other words, simulation can fill in the gaps created by clinical variability to ensure that all required competencies can be achieved and demonstrated.7 The simulation checklist was refined based on the observed performance of skills in the simulation, the required orientation competencies, and the patient care priorities at our hospital. The group workshop was also added to bridge the learning gap created by clinical variability.
New hires with experience showed a greater post score in time periods 2–4 compared to those without prior experience. This is likely because they could use their existing clinical knowledge and skills to adapt to the neonatal and pediatric environment. In addition, their previous simulation experience may have helped improve skill demonstration during the post-simulation. One study with ECMO specialists demonstrated that ECMO specialists with more than one year of experienced outperformed the specialists with less than one year of experience.10 The new graduate RTs may still have some educational or experience deficits that had not been addressed. In the curriculum updates, we included additional didactic content, but it may not have covered all the topics reviewed in the simulation scenarios. We anticipate further modifications to didactic content or simulation scenarios to meet the needs of the orientees.
Limitations
This was a retrospective analysis of the performance of new hires who progressed through an evolving orientation curriculum process. The primary goal was to provide orientation to newly hired staff. The simulation scenarios were specific to pediatric patient care experiences at our institution, and the competence-based checklist reflected institutional standards that may not be similar in other health care settings. Some new hires may have had previous simulation experience and felt more comfortable in the simulation environment. Additionally, the participants likely had varying clinical experiences during their orientation time that might have affected score improvement.
Three educators were on site to observe all simulation sessions. A primary educator calculated the score for each participant and consulted with the other educators after each scenario to provide input on the scoring. This suggests a high inter-rater consistency with the scoring. In the future, it would be important for educators to participate in formal inter-rater reliability training to maintain the level of quality and validation of the program.
The use of competence-based simulation assessment would be limited to those health care settings where simulation is an available tool. The cost of adding simulation assessment to an orientation program cannot be ignored. The simulation-based assessment requires a simulator, simulation space, educators, and a simulation operator. Cost analyses should be considered when adopting this in a health care setting.
Conclusions
Implementation of a competence-based simulation program customized the orientation to the needs of the new RT and helped improve the successful advancement of the new hire to ICU orientation. Simulation assessment scores increased between before and after orientation and may be used to guide decisions regarding readiness for orientation completion.
Acknowledgments
The authors would like to thank Natasha Lavin MSc RRT RRT-NPS CPFT, for her assistance and patience in editing this project.
The authors would also like to posthumously recognize Maureen Ginda for her tireless work and invaluable contributions in developing and facilitating our orientation program. Maureen had a wide breadth of respiratory care knowledge and experience, which she easily shared with all members of the health care team. She was an irreplaceable mentor to our staff, exhibiting how to treat each patient and their families with care, compassion, and respect. Maureen is greatly missed.
Footnotes
- Correspondence: Honey Pezzimenti MEd RRT RRT-NPS AE-C, Children’s Hospital of Philadelphia, Respiratory Care Department, 3401 Civic Center Boulevard, Philadelphia, PA 19104. E-mail: PezzimentH{at}chop.edu
Ms Napolitano discloses relationships with Dräger Medical, Vero-Biotech, Smiths Medical, Philips Respironics, and Actuated Medical. The remaining authors have disclosed no conflicts of interest.
A version of the study was presented by Ms Pezzimenti as an Editors’ Choice abstract at AARC Congress 2021 LIVE!, held virtually December 1, 2021.
Supplementary material related to this paper is available at http://www.rcjournal.com.
- Copyright © by 2022 Daedalus Enterprises
