Abstract
BACKGROUND: The modified early warning score (MEWS) is used to detect early clinical deterioration and to escalate care as needed. Respiratory therapists (RTs) usually do not use the MEWS even when it is implemented as a default in the electronic health record system. This study explored whether the technology acceptance model could predict the intentions of RTs to use the MEWS.
METHODS: A validated survey that uses a pretest/posttest design was used to determine the effect of an educational intervention (lecture and interactive small group session) on RTs' MEWS knowledge. We also measured key determinants of the intention by RTs to use the MEWS based on the constructs of the technology acceptance model. The survey was distributed to 75 RTs employed at a Midwestern academic medical center.
RESULTS: There was a 61% survey response rate. Statistical analysis of the survey data demonstrated that the educational intervention increased the MEWS knowledge score from 2.0 before education to 4.0 after education (P < .001). Moreover, there was a statistically significant increase in the behavioral intention score, from 3.0 before education to 4.0 after education (P < .001). Partial least squares structural equation modeling revealed that MEWS knowledge influenced perceived ease of use, which influenced attitude, which influenced behavioral intention.
CONCLUSIONS: Numerous studies have demonstrated that a change in behavioral intention is a good predictor of change in behavior. The increase in the RTs' knowledge, attitude, and behavioral intention scores after MEWS education indicated that these RTs may be more inclined to use the MEWS if they were educated about its clinical relevance and if their attitude toward using it were favorable. Analysis of the study results also indicated that the technology acceptance model could serve as a framework to guide respiratory care managers in the development of strategies to successfully implement new systems or processes that are intended to be used by RTs.
- modified early warning score
- respiratory therapists
- education
- technology acceptance model
- system implementation
- behavioral change
- user acceptance
Introduction
User acceptance of any new system is a complex phenomenon that continues to intrigue and perplex researchers and practitioners. Consequently, there is a plethora of studies from various domains that focus on explaining why people use or reject new systems. In the health-care sector, the proliferation of health information technologies over the past 2 decades has generated research aimed at discovering key determinants of successful implementation of these technologies.1–6 User acceptance studies in the health-care domain have used theories and models of motivation, decision-making, and technology acceptance to identify and measure key determinants of clinicians' actual use of health information technologies or intention to use health information technologies. Some of the most widely used theories in health information technology adoption studies include the technology acceptance model (TAM),7,8 the unified theory of acceptance and use of technology,9 and the theory of planned behavior.10 Moreover, it has been noted that, because beliefs affect attitude toward system use, pre-implementation beliefs of end users may influence the intention to use the system and actual system use after implementation.11
Health information technology user acceptance studies focus predominantly on physicians and nurses, and, to our knowledge, there is no study that has addressed respiratory therapist (RT) acceptance of new systems. This is a significant knowledge gap because RTs are integral members of the health-care team in North America and play a key role in implementing evidence-based care bundles aimed at preventing serious preventable iatrogenic complications, for example, ventilator-associated pneumonia. Moreover, because many care bundles are composed of protocols to decrease variation in care and to improve safety and quality of care, RTs routinely engage in protocol-directed care. However, multiple studies demonstrate that compliance with protocols by RTs and other clinicians is suboptimal.12–15
To ease clinicians' cognitive burden and improve compliance with protocols, there have been increasing efforts to automate protocols in computerized decision support systems.16–18 Walsh et al19 developed a computerized mechanical ventilation monitoring system capable of real-time identification of variances from pre-determined goals of mechanical ventilation. This system, and other decision support systems, could assist RTs in executing mechanical ventilation protocols, but it is not known if RTs would actually use them. Given the growing number of new and complex systems that RTs will be expected to use, it is important to understand factors that influence RT adoption of new systems.
A relatively simple warning system that can be used by RTs to enhance the quality of care they provide is the modified early warning score (MEWS). The MEWS is a validated scoring system that is used to detect clinical deterioration in hospitalized patients and has also been used to predict cardiac arrest,20 patient mortality rate, hospital admission from the emergency department, and the rate of ICU admission.21–25 It is possible for the MEWS score to be automatically generated by an electronic health record system. The MEWS requires the measurement of only 5 commonly measured physiologic variables and may be used in patients who are not considered to be critically ill.20 The components of this scoring system are the following: systolic blood pressure, heart rate, breathing frequency, temperature, and level of consciousness.21
The implementation of any new monitoring system in a hospital setting requires behavioral change. Health-care employees, as with other employees, are not comfortable with change, especially when behavior needs to be modified. Russ et al26 focused on identifying barriers and facilitators of implementing the World Health Organization surgical safety checklist across hospitals in England; a key finding was that a major barrier to implementing the checklist was individual resistance to change, particularly among high-ranking physicians. This finding was corroborated in a systematic review of qualitative evidence relevant to surgical checklist implementation.27 Factors such as cultural values, economic aspects, and individual personal preferences all affect individual behaviors.28 Consequently, using behavioral theories is beneficial for planning implementation of new systems or interventions.
Several studies used the TAM to measure the intention of health-care professionals to use a new system or technology after its implementation.29–36 The TAM posits that intention to use a new system is influenced by several factors, such as the usefulness of the system, attitudinal behavior, and the system's ease of use.37,38 (Fig. 1). Some of the studies30–36 that investigated the applicability of the TAM in predicting user acceptance of a new system demonstrated that multiple factors may affect the intent to use the system. These factors include subjective norms, job relevance, and habits that should be taken into consideration when studying the actual usage of the new system.32,39 The primary aim of our research was to discover key determinants of the intention of RTs to use the MEWS and to determine whether an enhanced TAM is an appropriate model for explaining RT intentions to use the MEWS. The external factors of awareness (knowledge of the MEWS), and job relevance of the MEWS were added to the original TAM to obtain the enhanced TAM that served as the research model for our study (Fig. 2).
The Technology Acceptance Model.
Research model. MEWS = modified early warning score.
QUICK LOOK
Current knowledge
Implementation of any new system is influenced by both organizational and human factors. The effect of human behavior, a critical success factor of system implementation, is more difficult to predict. This is a cause for concern in health care, in which many system implementation projects involve life-support systems. Several behavior change theories and models have been developed to provide a better understanding of an individual's response to change.
What this paper contributes to our knowledge
A validated survey based on the enhanced technology acceptance model was used to assess the intention of respiratory therapists (RTs) to use the modified early warning score (MEWS) before and after receiving education. Results of the study showed that RTs had a more favorable attitude toward using the MEWS, increased knowledge, and increased intention scores relevant to using the MEWS in the future. The technology acceptance model was an appropriate model for assessing RTs' perception and reaction to new a system and may help respiratory care managers develop strategies aimed at obtaining staff commitment to use new systems and processes.
Methods
Study Design and Population
Our study used a pretest/posttest design in which RTs employed at an urban academic medical center located in the Midwestern United States were recruited by using consecutive sampling, the most robust non-probability sampling technique. All practicing full-time and part-time RTs employed at the medical center were included in the study. RTs who worked through staffing agencies were excluded. Approval for the study was obtained from a local institutional review board.
Development of Survey Instrument
The instrument development process described by LoCurto and Berg40 guided the development and validation of the survey instrument. The hypothesized determinants of the intention of RTs to use the MEWS are based on the 3 constructs within the TAM as well as knowledge (awareness) and job relevance. Items used to measure each of these now 5 constructs as well as the behavioral intention construct were generated from a review of the information systems and health-care literature, and pretested by a sample of 36 RT students and RTs from health-care organizations, which did not include the main study site. This resulted in rewording of some of the survey items. Subsequently, the survey was sent to 3 topic and methods experts for item screening, who recommended removal and rewording of some items.
To determine whether survey items aligned distinctly with 1 of the now 6 constructs, principal components analysis with varimax rotation was conducted.41 For this analysis, a web-based format of the survey was distributed to 60 RT students and RTs from health-care organizations, which did not include the main study site, who were recruited by using purposive and snowball sampling. The suitability for conducting principal components analysis was assessed before analysis, which included having at least one correlation coefficient of >0.3, overall Kaiser-Meyer-Olkin measures between 0.7 and 0.8,42 and a significant Bartlett test of sphericity (P < .001). The data demonstrated that the questionnaire measured the intended constructs.
Subsequently, we conducted reliability and validity assessment of the instrument via partial least squares structural equation modeling by using XLSTAT 2017 for Excel (Addinsoft, Paris, France). Reliability analysis demonstrated that the scales had a high level of internal consistency, as determined by composite reliability scores, which ranged from 0.83 to 0.95. Convergent validity assessment of the survey instrument revealed that the items used to measure a specific construct loaded higher on that construct (loadings of >0.50) than on any other construct. In addition, discriminant validity assessment was determined by demonstrating that the square root of the average variance extracted for each construct was higher than the construct's correlation with any other construct (Fornell-Larcker criterion).43 The survey instrument, available as an online supplement (see the supplementary materials at http://www.rcjournal.com) shows the items used to measure each construct.
Study Procedures
The final survey instrument was embedded in REDCap (ProQuest, Ann Arbor, Michigan), a secure web-based electronic survey tool and research database, and was distributed to all 75 RTs employed at an urban academic medical center through a web link to the department's e-mail list. None of these RTs had participated in the exploratory assessment of the survey instrument. The survey was deployed for 2 weeks, and, at the end of 1 week, a reminder was sent to potential participants. An educational session on the MEWS was then provided to the RTs. The educational session delivered via lecture and interactive small group session included a general overview of the MEWS, its components, the purpose of the MEWS, how to classify the severity of a patient's clinical deterioration by using MEWS scores, and possible RT interventions that could be included in the escalation of care. We also taught the RTs how to add the MEWS score column to their patient lists in the electronic health record system. Twenty-four hours after the educational session, the survey was redistributed to the RTs. The survey was deployed for a 1-week period, with reminders given to the RTs during daily shift reports. Redistribution of the survey was to determine if there was any change in MEWS awareness after the MEWS educational session.
Data Analysis
Median scores were computed for the constructs included in the research model, namely, awareness (MEWS knowledge), job relevance, perceived ease of use, usefulness, attitude, and behavioral intention. A Wilcoxon signed-rank test was used to determine if there was a significant difference between the median scores for each construct before and after the educational intervention and tested at alpha = 0.05. The model was subsequently analyzed by using partial least squares structural equation modeling to discover the key determinants of the intention of RTs to use the MEWS and to determine whether the enhanced TAM is an appropriate model for explaining their intention to use the MEWS. Assessment of the relationship between the variables of the research model was performed by calculating path coefficients (a measure of the strength of the relationship between variables) and their t-values as well as the R2 value by using XLSTAT 2017 for Excel (Addinsoft). The R2 is a measure of the proportion of variance of the dependent variable (eg, behavioral intention) that is explained by the independent variable (eg, attitude or perceived usefulness).
Results
The survey was completed by 46 of the 75 RTs (61%) employed at the study site. A summary of the demographic characteristics of the survey participants is presented in Table 1. Of the 46 participants recruited to the study, the MEWS educational session elicited an increased knowledge score in 45 participants and a decreased score in one participant. In addition, the behavioral intention score post-educational intervention increased in 30 participants compared with the behavioral intention score pre-educational intervention; in 11 participants, there was no improvement in the behavioral intention score; and 5 participants had a lower intention score post-educational intervention. A Wilcoxon signed-rank test determined that there was a statistically significant median increase in the MEWS knowledge score, of 2.0, post-educational intervention (4.0) compared with the pre-educational intervention (2.0), P < .001. There also was a statistically significant median increase in behavioral intention score of 1.0 post- educational intervention (4.0) compared with pre-educational intervention (3.0), P < .001.
Relationship Between Demographic Variables and the Median BI Score Before and After MEWS Education
With regard to the relationships between the enhanced TAM constructs before the educational intervention, the perceived ease of use was found to have a significantly positive effect on the perceived usefulness of the MEWS based on results from the partial least squares structural equation modeling analysis. In fact, the perceived ease of use and job relevance were found to explain 73% of the variance (adjusted R2 = 0.73) (P < .001 and P = .002, respectively) in perceived usefulness. In addition, perceived ease of use was found to have a significantly positive effect on attitude (adjusted R2 = 0.63, P = .001) and attitude was found to have a significantly positive effect on behavioral intention (adjusted R2 = 0.85, P < .001). Although the TAM posits that perceived usefulness affects attitude toward a new system or tool and also directly influences behavior, these relationships were not observed in the pre-educational intervention data. A summary of these findings appear in Table 2 and Figure 3.
Results of MEWS Pre-Education Relationships Among Enhanced TAM Constructs
Summary of research model analysis (pre-education). MEWS = modified early warning score.
Consistent with the TAM postulates, both perceived usefulness and perceived ease of use were found to have a significantly positive effect on attitude, and attitude was found to have a significantly positive effect on behavioral intention post-education. However, no statistically significant effect of the perceived ease of use on perceived usefulness was observed, and neither was there a statistically significant effect of perceived usefulness on behavioral intention (Table 3 and Fig. 4). Overall, the enhanced TAM model seemed to be appropriate for explaining the intention of the RTs to use the MEWS based on the results of the partial least squares structural equation modeling analysis as the adjusted R2 = 0.85, P < .001 pre-educational intervention and the adjusted R2 = 0.74, P <.001 post-educational intervention. This meant that the enhanced TAM model explained 74–85% of the variance in the intention of the RTs to use the MEWS.
Results of MEWS Post-Education Relationships Among Enhanced TAM Constructs
Summary of research model analysis (post-education). MEWS = modified early warning score.
Discussion
Our study found that the enhanced TAM was an appropriate model for predicting the intention of the RTs to use the MEWS. This finding was consistent with other studies that investigated the effectiveness of the TAM in predicting technology acceptance among physicians and nurses.8,30,31,44,45 Moreover, our finding that attitude was a strong predictor of behavioral intention was supported by the findings of Hsiao and Chen8 with regard to the critical role of attitude in predicting intention to use computerized clinical practice guidelines by physicians. Both external variables added to the enhanced TAM (ie, knowledge, and job relevance) positively contributed to the TAM variables of perceived usefulness and perceived ease of use pre- and post-educational intervention. This was consistent with evidence with regard to the direct effect of underlying factors on the perceived ease of use and usefulness of a new system.35–44 For example, Dünnebeil et al44 found that the intensity of technology utilization had a highly significant effect on the perception of usefulness of a new system. In addition, the processes of orientation as well as E-health knowledge had a significant effect on the perceived ease of use of the E-health technology system.
In this study, after the implementation of the educational session, most of the participants demonstrated statistically significant improvement in knowledge, attitude, and behavioral intention scores relevant to the use of the MEWS. Our findings were consistent with a cross-sectional study31 that measured nurses' acceptance and intention to use a mobile electronic medical record in which the investigators concluded that multiple factors may positively or negatively affect staff attitude toward using a new system. The finding in our study that some participants scored the same or even less regarding their intention to use the MEWS after the educational session could be explained by different factors, such as feelings of discomfort or insecurity, whereas those with a higher behavioral intention score after the educational session could be more optimistic and may be early adopters of innovation.38 In addition, the results could have been affected by some concerns that were raised by the staff regarding not feeling comfortable with using a scoring system to assess patients when it is not currently used by other clinicians such as physicians and nurses who are considered to be vital participants in patient care. Based on their misgivings regarding endorsement of the MEWS by key members of the health-care team, the RTs verbalized that using the MEWS could result in wasted effort.
Our finding that perceived usefulness had a significant effect on attitude after educational intervention was supported by the findings of Aggelidis and Chatzoglou,34 who reported a strong direct effect of perceived usefulness on attitude in their study of health information technology acceptance by administrative, medical, and nursing personnel in Greece. We also found that perceived usefulness did not significantly influence behavioral intention pre- or post-educational intervention, which is not consistent with this specific postulate of the TAM, nor with the results of a systematic review of the usefulness of the TAM in predicting physician acceptance of health information technologies.30
This difference observed in our study may be attributed to the opinion of the RTs that, if key members of the health-care team are not currently using the MEWS, then it may be inappropriate for them to incorporate it into their patient assessment. Indeed, Hsiao and Chen8 found that social influence and organizational support significantly influenced physician intention to use computerized clinical practice guidelines. However, there were some RTs who showed enthusiasm and an overall positive attitude toward using the MEWS. This was consistent with findings that demonstrated the positive effect of specific personal traits, including curiosity and openness to new experiences on behavioral intention.32 Social influence and training could have a secondary impact on behavioral intention.34 In our study, the educational sessions were structured to facilitate small group or even one-on-one training to assure the full understanding of the staff and to increase the likelihood that all their questions would be answered. This could have contributed to the improvement seen in the knowledge score after education.
According to the diffusion of innovation theory by Rogers,46 there are multiple factors, such as the type of innovation, channel of communication, and social norms, that may affect the adoption of a new system. Consequently, to increase the likelihood of RTs adopting new technology, communication about the technology should occur not only among the staff but also among the leaders of different clinical departments. This would augment the diffusion of the concept and contribute to more-widespread adoption of technology. Social influence may have a dual effect because it could positively affect adoption of a new tool or system when individuals believe that people who are important to them (eg, supervisors or respected colleagues) endorse the system. Conversely, social influence could negatively affect adoption of the new tool or system when individuals perceive it as a mandate with which they do not want to comply47,48 Consequently, to foster positive social norms toward the system, it is important to have super users or champions of the system who are well-respected colleagues of the system's intended users.1,8
We conducted this study to test whether the enhanced TAM could help us predict the use of the MEWS by RTs in an academic medical center. Our findings indicate that providing education to RTs about the MEWS increased scores for perceived knowledge, job relevance, attitude, usefulness, and ease of use. Moreover, we found that, consistent with a central hypothesis of the TAM, attitudes toward using the MEWS positively affected behavioral intention to use it.
There were some limitations to our study that are inherent in research involving self-reported data. Such limitations included response bias attributable to participants being confused about some of the questions, recall bias, and common rater effect, which refers to artificial covariance between the predictor and outcome variable because responses to all survey items were obtained from a single individual. Also, consistency motif, which refers to the tendency for respondents to maintain consistency in their responses to questions, was another potential source of error. Another limitation to this study was that it was done in one hospital only, with a small non-randomly selected sample of RTs. Moreover, the study participants were relatively young with predominantly advanced degrees. In the department, 67% of the RTs had ≤10 y of clinical experience and most had a baccalaureate or a higher academic degree. Consequently, results cannot be generalized to all RTs. Further, the study addressed implementation of only one form of technology and did not test whether the enhanced TAM applied to other health information technologies.
Conclusions
Numerous studies over several decades demonstrated that change in behavioral intention is a good predictor of change in behavior. Consequently, RTs may be more inclined to incorporate the computer generated MEWS in their assessment of patients if they are educated about its clinical relevance and if their attitude toward using it is favorable. Further research is required to measure the relationship of behavioral intention to the actual use of specific technologies, for example, computer generated MEWS, and to confirm how closely TAM constructs relate to each other. The study results also indicated that the enhanced TAM could serve as a framework to guide respiratory care managers in the development of strategies to successfully implement new systems or processes intended to be used by RTs and other health-care providers. However, a study with a larger sample of RTs with more diverse demographic characteristics is needed to provide more convincing evidence regarding the ability of the enhanced TAM to predict RT adoption of new systems.
Acknowledgments
The authors thank the staff and management of the Respiratory Care Department, Rush University Medical Center, for facilitating this study.
Footnotes
- Correspondence: Constance C Mussa PhD RRT RRT-NPS, Rush University, AAC 765A, 600 S Paulina St, Chicago, IL 60612. E-mail: constance_mussa{at}rush.edu.
The authors have disclosed no conflicts of interest.
Supplementary material related to this paper is available at http://www.rcjournal.com.
- Copyright © 2019 by Daedalus Enterprises
