Abstract
BACKGROUND: Learning bronchoscopy is challenging for novices, as it requires navigation in a 3-dimensional space under 2-dimensional viewing conditions and execution of complex motor skills with an unfamiliar instrument. Mental practice exercises are based on repeated visualization of motor actions without physically performing them, thereby promoting the learning of skills. We aimed to evaluate whether a teaching intervention including mental practice exercise modules for the acquisition of bronchoscopy skills improves fiberoptic intubation performance of novice learners.
METHODS: In this prospective cohort study, 24 pediatric intensive care trainees and respiratory therapists participating in a bronchoscopy learning curriculum in 2016–2017 attended a theoretical lecture followed by self-guided learning. Subsequently, the learners were randomly assigned to either participating in a teaching intervention including mental practice exercises or not (control group). The primary outcome was time to complete their first bronchoscopic intubation using a virtual reality simulator. Secondary outcomes were the occurrence of “red outs” (ie, the anatomy could no longer be visualized) or collisions with the airway wall. Bayesian Poisson Mixture models were used to estimate the effect of the intervention on outcomes. Furthermore, participation in the teaching intervention was examined in short interviews and with descriptive thematic analysis.
RESULTS: Subjects in the intervention group completed the bronchoscopy on average 1.2 times faster (rate ratio 1.2 [95% credible intervals 1.1–1.3]). The posterior probability that the teaching intervention reduced the occurrence of “red outs” by more than half was 86%. No differences were found regarding the odds of colliding with the airway wall. Everyone except 1 trainee in the mental practice group engaged with and found the mental practice modules helpful.
CONCLUSIONS: A teaching intervention including mental practice exercises represents a valuable additional learning strategy promoting the performance and complex skill acquisition of novice learners in the initial stages of learning bronchoscopy procedures.
Introduction
Bronchoscopy is a technique to visualize the patient’s airways and is utilized daily by many medical specialists such as anesthesiologists, intensivists, and respiratory therapists. Training in bronchoscopy skills uses simulation practice to augment the clinical experiences.1-3 For bronchoscopy training, simulation specifically involves either a virtual reality (VR) simulator or an inanimate airway model with a flexible bronchoscope.1,4-6 A major issue with simulation is cost. VR simulators are expensive; the inanimate airway model, though less expensive, requires a real flexible bronchoscope and a tower. The use of these instruments in simulations may not be possible for 2 reasons: first, most institutions have only a bronchoscope/tower designated for patient care; second, there may be concerns about the expense of maintenance and repair, given the potential for damage of this fragile instrument in the hands of novices. Indeed, the yearly cost of bronchoscope repair quoted in the literature is close to $20,000. Importantly, 50–78% of that cost is incurred by inexperienced and inattentive users.7,8
Novice learners often experience difficulties manipulating a real or VR flexible bronchoscope because it requires the execution of complex psychomotor skills with multitasking, navigating a 3-dimensional space under 2-dimensional viewing conditions, and using a counterintuitive instrument in that the tip of the bronchoscope moves in the opposite direction of the human hand. Most of these skills, and especially the combination of these factors, are unique to the flexible bronchoscope.
The first stages of learning a new skill are of a primarily cognitive nature.9,10 Task-specific mental representations play an important role in guiding the motor organization during the realization of a movement directed to attain a specific goal.11-13 Mental practice is the cognitive rehearsal of a task in the absence of overt physical movement. Mental practice in the form of imagery rehearsal, repeatedly simulating (imagining) an action without executing it, is known to improve performance and promote learning by forming these mental representations.14-17 In addition, skillful coordination occurs when appropriate mental representations of the motor task and action goals are constructed. On the basis of these motor-learning theories, one can speculate that novice learners may benefit from “pre-training” on simplified models that enable mental practice and consequently the formation of initial mental representations for bronchoscopy tool-handling skills. Observing and interacting with simple models can achieve this, so long as the models contain the fundamental constructs of the skills.18,19 Once mental representations are formed, the learners will benefit more from interacting with real scopes, shortening the expensive learning curve and potentially minimizing damage to equipment.
On the basis of these theoretical premises and the need for a simplified model of training to allow novice learners to develop their instrument holding and navigation skills, a short training module implementing mental practice for bronchoscopy training has been created. In this study, we asked whether novice learners would participate in training modules that target mental practice and if undergoing a mental practice exercise would subsequently result in better performance in a VR simulator setting, as measured by the time to complete a bronchoscopic intubation and by the occurrence of “red outs” (ie, the anatomy could no longer be visualized) or collisions with the airway wall during the intubation.
QUICK LOOK
Current Knowledge
Fiberoptic intubation is an important skill used by many specialty care providers. Simulation training in addition to clinical experience is used to teach the skill to trainees. In the current models of training, novice learners face challenges due to complexity of this skill and the fragility of instruments.
What This Paper Contributes to Our Knowledge
The first stages of learning a new skill are of a primarily cognitive nature. Mental practice (ie, the cognitive rehearsal of a task in the absence of overt physical movement) is known to improve performance and promote learning by forming the mental representations. A teaching intervention including mental practice exercise modules before training with a virtual reality bronchoscopy simulator facilitated the initial learning of skills as portrayed by fiberoptic intubation performance and learners’ comments.
Methods
Design, Setting, and Subjects
For this prospective cohort study, subspecialty residents in pediatric critical care and respiratory therapists working in the Department of Pediatric Critical Care Medicine, Hospital for Sick Children, Toronto, in 2016 and 2017 were invited through e-mail and advertisements in the trainees’ and respiratory therapists’ lounge to participate in a bronchoscopy-learning curriculum. Respiratory therapists assist in bronchoscopic procedures and training of the learners in our institution. The curriculum consisted of 3 parts, with learners assigned to either the mental practice group (ie, the intervention group) or the control group during the second part of the curriculum. All residents and respiratory therapists who declared in their consent that they had never performed a bronchoscopy procedure (ie, confirmation that they are true novices) were enrolled in the study, which was approved by the institution’s Research Ethics Board. A schematic illustration of the study design and profile are provided in Figure 1.
Flow chart. VR = virtual reality.
Bronchoscopy Learning Curriculum
During the first part of the curriculum, residents and respiratory therapists attended an instructor-led group lecture that provided an overview of the procedure (ie, consent, preparation, indication, and complications) as well as an introduction to the anatomy, including normal and abnormal findings. In addition, the instructor demonstrated a full procedure of bronchoscopic intubation and navigation of the airway. Following the theoretical lecture, residents and respiratory therapists practiced self-guided learning of the anatomy using an eLearning site (http://www.thoracic-anesthesia.com/?page_id=2. Accessed March 20, 2021). The second part of the curriculum involved familiarization with the bronchoscopy handling tool. Residents and respiratory therapists were assigned to either the standard curriculum (ie, control group) or to the mental practice group (ie, intervention group) using computer-based block randomization with random variable block sizes. Trainees randomized to mental practice training were allowed 5–10 min to practice with self-directed mental practice modules. In contrast, the trainees enrolled in the standard curriculum underwent an instructor-led group discussion about handling the bronchoscope and the VR simulator. Considering the criticism of educational studies that compare the effect of receiving a teaching intervention to not receiving a teaching intervention, we chose to compare 5–10 min of self-directed mental practice to a standard teacher-led curriculum.
The third part involved practice in a VR simulator. We compared the difference in performance between the intervention group and the control group during their first fiberoptic intubation. The remainder of the curriculum continued as planned with instructor guidance, reaching the goals and supervision at the bedside.
Intervention
Trainees assigned to the intervention group received training with mental practice modules, while the trainees in the control group participated in an instructor-led group discussion on the handling of the bronchoscope and functioning of the VR simulator.
Navigation with a bronchoscope is achieved with a combination of 4 actions: rotation, moving forward and backward, moving upward and downward, and suctioning to clear the airways. One approach to mental practice includes scripts outlining the steps of a task to teach psychomotor skills. These scripts contain detailed instructions on how to perform a task as well as prompts from different sensory modalities that augment trainees’ experience to facilitate more accurate mental representations: visual (imagining oneself performing the procedure), cognitive (thoughts guiding the procedure), and kinesthetic cues (what it feels like to execute a particular action).20 During the first stage of the mental exercise, the trainees of the intervention group watched a slide show with different pictures of the respiratory tract. Each slide presented a single action in a successful bronchoscopy maneuver, as opposed to a video showing the combination of many actions. This decomposition into single actions provided the trainees with detailed instructions on the optimal movement that the trainees should perform during a bronchoscopy procedure. After this first stage, the trainee imitated the simple actions required to navigate the airway using a bronchoscope imitator that included all 3 features required during navigation of the airway (ie, suction button, lever, and flexible tubing), but it did not respond to any of the trainees’ actions. This intervention was intended to provide kinesthetic cues. The tutorial gradually advanced from instructions for individual actions to more complex movements requiring 2 or 3 simultaneous actions (eg, rotating left and moving forward). Lastly, images were shown without instructions, requiring the trainees to decide on the necessary manipulations, thereby facilitating the formation of a mental representation of the skill. The training module was piloted in 5 novice trainees to explore whether trainees would engage in this type of practice and for how long. Based on the results, we determined that the trainees would engage for 5–10 min. The 5 trainees were not part of subsequent testing and randomization.
Quantitative Outcomes
As primary outcome, we decided to measure performance during the first bronchoscopic intubation in a VR simulator. Secondary outcomes were the occurrence of red outs or collisions with the airway wall. The time to successful intubation is of utmost clinical importance given the high stakes of the procedure. Furthermore, other clinically relevant clinical outcomes, such as performance during navigation of the airway, might be confounded by the process of learning to intubate the airway (ie, conditional dependence). Navigation of the airway for a novice trainee at their first attempt takes longer than the time spent for the mental practice module. Therefore, learning through practicing to intubate in the simulator might substantially bias effect estimates measured during navigation of the airway. Subsequently, the remainder of the curriculum continued as planned with instructor guidance, reaching the goals and supervision at the bedside, targeting bronchoscopic intubation and navigation of the airways.
The performance of a fiberoptic intubation was evaluated on the Virtual Reality Bronchoscopy Simulator (EndoVR Bronchoscopy Simulator; CAE Healthcare, Montreal, Quebec, Canada). The VR simulator automatically scores the duration of each procedure, the number of collisions with the airway wall, and the duration of red outs. Currently, investigators have an array of techniques for measuring bronchoscopy skills at their disposal, including the Ontario Bronchoscopy Assessment Tool and web-based resources, especially for assessment of performance at the bedside. We chose the VR automatic scoring because several studies have examined the validity of this scoring system on its own, showing correlational validity evidence (ie, ability to discriminate between novice, intermediate, and experts and score improvement with repetitive practice), and they are sensitive enough to detect learning changes.21-24
Qualitative Outcomes
In the present work, the principal investigator (BM), with experience in qualitative interviewing, conducted 5–10 min face-to-face interviews with novices in the intervention group regarding the challenges of learning a complex skill and the experience with and impact of mental practice to examine the engagement of trainees in this curriculum (see the supplementary materials at http://www.rcjournal.com).
Statistical Analysis
For quantitative analysis, we investigated the effect of the mental practice intervention on the duration of intubation compared to the control group using a Bayesian Poisson model. Odds ratios for the occurrence and rate ratios for the duration of “red outs” during the bronchoscopy procedure in the virtual reality simulator were estimated using a Bayesian Poisson mixture model, adjusted for the duration of the bronchoscopy. The same model was used to compare the odds and rate of collisions with the airway wall between subjects of the mental practice group and the standard curriculum group, adjusted for the duration of the bronchoscopy. Model estimates and 95% credible intervals (CrIs) (ie, 2.5th and 97.5 percentiles of the posterior distribution) were derived using Markov chain Monte Carlo sampling (3 chains with 20,000 iterations burn-in and 20,000 saved iterations per chain). Convergence of the models was verified with Gelman-Rubin statistics. We used a Bayesian framework, in contrast to traditional frequentist statistics, for the reliable estimation of odds ratios and rate ratios accounting for a smaller cohort size.25,26 All analyses were conducted in R 3.6.3 and JAGS 4.3.0 (see the supplementary materials at http://www.rcjournal.com).
For the qualitative analysis, subjects’ interviews were recorded, transcribed verbatim, and analyzed using qualitative descriptive methods.27 Initial coding and categorizing to identify themes and subthemes were done by the principal investigator (BM) and occurred concurrently with data collection. There were no new themes that needed to be explored, and the interview guide remained unchanged. After the initial analysis was performed, the findings were discussed with the second team member (GDL), who read all of the transcripts and identified the same themes.
Results
Twenty-four residents and respiratory therapists consented to participate in the study, and data from all 24 subjects were used. None of the subjects opted out of being interviewed or completing the exam of fiberoptic intubation in the virtual reality simulator. Subjects’ scores are described in Table 1.
Subject Scores on the Virtual Reality Simulator
Fiberoptic Intubation Scores in Virtual Reality Simulator
The mean ± SD time to complete the intubation for the mental practice group and the standard curriculum group was 106.5 ± 42.4 s and 127 ± 44.8 s, respectively (Fig. 2A). Subjects in the intervention group completed the bronchoscopy on average 1.2 times faster than those in the control group (rate ratio 1.2 [95% CrI 1.1–1.3]). The intervention group subjects also had 0.3 times lesser odds of having a red out compared to those in the control group (odds ratio 0.3 [95% CrI 0.1–0.7]). The posterior probability that the intervention reduced the occurrence of red outs by more than half was 86%. Adjusted for the duration of the bronchoscopy, the duration of red outs was 0.5 times shorter in subjects of the intervention group compared to the control group (rate ratio 0.5 [95% CrI 0.4–0.6]). The posterior probability that the intervention reduced the amount of time spent in red out by more than half was 65% (Fig. 2B). No difference was found regarding the odds (odds ratio 1.0 [95% CrI 0.3–1.0]) or rate of collisions (rate ratio 1.0 [95% CrI 0.7–1.3]) with the airway wall between intervention and control group, adjusted for the duration of the bronchoscopy (Fig. 2C).
Performance recorded during a first bronchoscopy using a virtual reality simulator. After an instructor-led lecture and subsequent self-guided learning of the anatomy, we compared trainees who completed additional mental practice exercise modules (intervention group) with trainees who attended an additional group discussion and orientation session on the virtual reality simulator (control group).
Thematic Analysis of the Interviews
Interviews with trainees lasted approximately 5–10 min. All trainees expressed difficulties with maneuvering the bronchoscope initially due to the complexity of movements required to navigate the airway tree. All trainees participated in all stages of mental practice and liked the progression from a single action to simultaneous actions to deciding the necessary movement, depending on the image of respiratory tract seen on the screen. All trainees expressed confidence in the improvement of their skill levels but indicated that they would not try the procedure at the bedside without supervision.
All but 1 subject in the intervention group found the mental practice helpful. That trainee did not like the lack of haptic feedback in the bronchoscopy exercises.
Yeah, it doesn’t tell you if you’ve done things right. It just tells you and you think about what to do but the response is not to your action. For things that are manual I need the practice and the feedback. . . and the immediate, and the way it feels, and so that’s how I learn. (L3)
Other trainees felt that the pre-training made it easier for them to navigate in the VR simulator and knowing what actions to do to be able to intubate and inspect the airway.
It was enough as a middle step. It gave you an orientation like which is up, which is down, how to move the tip of the bronch or which way to rotate to get where you want to go. (L1)
I find it very useful because that gives you a little bit of preparation because once you’re on the VR simulation you feel overwhelmed. . . I was more comfortable, using the bronch, because I knew what to do. So I like the middle stage. (L8)
The introductory PowerPoint going once through the movement required was good because I had no idea about those movements. (L6)
I found it useful, basically the PowerPoint. It made me get acclimatized to the bronch movements. (L11)
Discussion
In this study of novice learners attending a bronchoscopy learning curriculum, we demonstrated that learners are engaged in a teaching intervention that allows mental practice to promote formation of initial action representation. We found that trainees performed their first bronchoscopic intubation in a VR simulator more quickly and spent less time in red outs if they had participated in a short mental practice exercise. An operator spends time in red out when the anatomy is not successfully visualized and difficulties are experienced in navigating the bronchoscope. We did not observe a difference in the number of collisions with the airway wall between the 2 groups, but the time taken to complete a fiberoptic intubation is of paramount clinical importance. Previous work has reported that practice in a VR setting can lead to acquisition of complex real-life skills.28-30 Our teaching intervention represents a simple cognitive exercise using a slide show tutorial with an instructional design that substantially improved the trainees’ performance and can easily be integrated in existing training programs to promote psychomotor skill acquisition of novice learners.
Mental practice has not been widely studied in medical education for skills training and may have a role in the initial stages of teaching.20,31,32 Motor learning theories propose that the first stages of skill learning are primarily cognitive in nature, and we might expect that changes evoked by mental practice (a cognitive type practice) would help when learning a new motor skill.15 There is also analogy at the cortical level between the mechanisms that mediate action observation and those involved in action execution.15 Mental representations underlie action understanding: the capacity to achieve the internal description of an action and to use it to organize appropriate future behavior.33 The cognitive mechanisms controlling skill execution develop over the course of learning. Skill acquisition is accompanied by both overt changes (performance improvement) and covert changes (cognitive improvement over time), therefore novice learners benefit from a short initial teaching intervention including mental practice exercises that promote the development of mental representations leading to an improvement in their performance.33-35 While deconstructing the skill and forming mental representation through mental rehearsal may help, there are also data illustrating that learning might be enhanced through complexity and “desirable difficulties,” and one might speculate that dealing with challenges and difficulties while learning represents an integral part of the skill acquisition process itself.36 Mental practice exercise modules may facilitate skill acquisition of complex motor tasks through a better understanding and memory of specific aspects of the task. Our study investigated whether a teaching intervention including mental practice exercise modules specifically facilitates learning the execution of the task. Nevertheless, in real-life practice additional factors and performance pressure can substantially impact the ability of the learners to successfully perform the procedure at the bedside. Therefore, our trainees were provided with the opportunity to continue learning the task during supervised bedside practice after successfully completing the simulation training.
Our study had several limitations. We measured the performance of the trainees using a VR simulator and not during a real-life exam. Fiberoptic intubation in a real-life setting requires skills beyond the simple knowledge of how to operate a bronchoscope. Therefore, measuring the effect of our intervention in a real-life patient intubation scenario following only didactic teaching or a mental rehearsal for novices might introduce substantial confounding and is at this stage of training experience questionable from an ethics perspective. The outcomes measured in the VR simulator have good validity, and previous work demonstrated successful translation from VR practice into acquisition of complex real-life skills.28-30 Furthermore, our study had a small sample size, which we addressed by using a Bayesian framework instead of a traditional frequentist approach.25 This sample size is well within the range used by other studies in the field of bronchoscopy training that investigated the same outcome measures.21,22 Lastly, we did not follow the trainees at the bedside to investigate how skills gained in simulation translated to real-life practice.
Conclusions
Teaching interventions including mental practice exercise modules cannot replace hands-on training with a bronchoscope, but they represent a promising additional learning strategy in preparing novice learners for their next steps in simulated and supervised clinical contexts.
Acknowledgments
We thank Adam Dubrowski PhD, for his feedback in the early stages of this project.
Footnotes
- Correspondence: Briseida Mema MD, Department of Critical Care Medicine, The Hospital for Sick Children, 555 University Avenue, Toronto, ON, M5G 1X8, Canada. E-mail: briseida.mema{at}sickkids.ca
Dr Mema presented at the International Association for Medical Education (AMEE) 2017 meeting, held August 26–30, 2017, in Helsinki, Finland.
Supplementary material related to this paper is available at http://www.rcjournal.com.
The work was supported by the Peri-operative Services Innovation Grant, The Hospital for Sick Children, Toronto, Canada (BM) and (MU) is supported by a Vanier Canada Graduate Scholarship, Canadian Institutes of Health Research (CIHR). The authors have disclosed no conflicts of interest.
- Copyright © 2021 by Daedalus Enterprises
