Abstract
BACKGROUND: The use of YouTube for providing medical information is increasing, and patients with COPD are likely to use digital media to obtain information for their conditions. This study aimed to analyze the reliability and quality of the videos that were most frequently viewed by patients with COPD who searched YouTube for information on pulmonary rehabilitation.
METHODS: A YouTube search was conducted by using the keyword “pulmonary rehabilitation.” All the videos were categorized into 2 groups according to source and purpose. For evaluating the reliability and quality, the modified DISCERN score and the pulmonary rehabilitation for COPD-specific score (PRSS) were used, respectively. Differences of the variables between the groups and associations across the groups were examined. The influence of the variables of the videos on the engagement of viewers and viewership was analyzed.
RESULTS: In total, 62 videos were analyzed. Most of the videos (54/62 [87%]) were uploaded by professionals and the remainder (8/62 [13%] by non-professionals. According to their purposes, 35 of 62 (56%) were informational and 27 of 62 (44%) were non-informational. The mean ± SD modified DISCERN score and the mean ± SD total the PRSS were 3.76 ± 1.04 and 4.13 ± 3.73, respectively. When comparing the videos according to professionalism, significant differences only in the modified DISCERN scores were observed. The engagement of viewers, viewership, mean modified DISCERN, mean total of the PRSS were significantly higher in the informational group than in the non-informational group. Video duration was positively correlated with engagement of viewers and viewership.
CONCLUSIONS: Popular YouTube videos on pulmonary rehabilitation for COPD were reliable; however, they were biased and of low quality in terms of providing content. Given YouTube’s advantage of easy accessibility and disadvantage of vulnerability to poor-quality information, professionals should pay more attention to uploading comprehensive high-quality videos for informational purposes.
- chronic obstructive pulmonary disease
- education
- internet
- pulmonary rehabilitation
- rehabilitation
- social media
Introduction
Pulmonary rehabilitation is a comprehensive, patient-tailored intervention provided with patient assessment, and it entails exercise training, education, and psychosocial support to improve the physical and psychosocial well-being of people with chronic respiratory disease.1 Pulmonary rehabilitation has been shown to substantially improve symptoms, exercise performance, and health-related quality of life in patients with COPD.1,2 Thus, pulmonary rehabilitation for COPD has become an important treatment option as an adjunctive therapy combined with conservative medical therapy.1 However, limited access to center-based pulmonary rehabilitation programs, despite the large number of patients with COPD, has led professionals to devise alternatives for the efficient provision of pulmonary rehabilitation awareness.3-6 Many previous studies show that telerehabilitation by using online videos is similar or superior to center-based pulmonary rehabilitation in terms of cost-effectiveness, awareness of diseases, and clinical improvement.7-10 However, according to other reports, patients with COPD are likely to use digital media to obtain medical information for their conditions.11,12
YouTube (Google, Mountain View, California), founded in February 2005, is a user-generated video search engine. With the rapidly increasing use of smartphones, YouTube has become one of the most visited web sites worldwide.13 YouTube has the advantage of allowing producers to freely upload videos, which allows viewers to watch them at no cost, and of having 2-way communication between producers and viewers. However, it also has the disadvantage of not offering a peer-review system to evaluate the reliability and quality of the videos. Based on these shortcomings, several studies have examined the quality of information on YouTube for various pulmonary diseases. However, the results of these studies have been inconclusive.14-17
Given the increasing use of YouTube for providing and accessing medical information and the possibility of being used as part of telerehabilitation and the varying results of previous studies on the usefulness of YouTube videos, it is necessary to evaluate the reliability and quality of YouTube videos on pulmonary rehabilitation for COPD.18 However, to date, to our knowledge, no studies have examined the current state of YouTube videos on pulmonary rehabilitation for COPD. Therefore, we conducted this cross-sectional study to analyze the reliability and quality of the videos that were most frequently viewed. Our findings could provide preliminary information to individuals who intend to upload videos on this topic.
QUICK LOOK
Current knowledge
Telerehabilitation by using online videos is similar or superior to center-based pulmonary rehabilitation in terms of cost-effectiveness, awareness of diseases, and clinical improvement. Patients with COPD are likely to use digital media to obtain medical information for their conditions. YouTube, one of the most visited web sites, has the disadvantage of not offering a peer-review system to evaluate the reliability and quality of the medical videos.
What this paper contributes to our knowledge
Overall video reliability of pulmonary rehabilitation videos for COPD was fair; however, most videos failed to provide sufficient information or provided misinformation. The videos produced by the professional group who post the most videos exhibited significantly higher reliability scores; however, these videos were not of good quality.
Methods
Video Selection
A cross-sectional YouTube search was conducted on June 26, 2021, by using the keyword “pulmonary rehabilitation,” which was chosen with author consensus. We used the view filter, which displays results in the order of the number of views, to analyze the most popular videos. On the assumption that no user would go beyond the first 5 pages (20 videos per page) for a specific search keyword, only videos on these pages were reviewed and evaluated. All commercials presented by YouTube at the beginning and end of the videos were ignored. YouTube videos that were evaluated for this study were accessible to everyone. Ethics committee approval was not required because this study did not include any human participants. The exclusion criteria were as follows: duplicate videos, non-English videos, clearly irrelevant videos, and videos of poor quality. Videos that deal with rehabilitation for patients with diseases other than COPD, such as coronavirus disease 2019 related pulmonary disease, idiopathic pulmonary fibrosis, and rehabilitation after spinal cord injury and lung transplantation, were also excluded. The remaining videos were then included for further analysis. A flow chart of the video selection is shown in Figure 1.
Flow chart.
Data Collection and Assessment
Characteristics and Categorical Distribution.
The following descriptive characteristics for each video were collected: title, affiliation of the uploader, number of views, number of likes, number of dislikes, number of comments, duration (min), and posting days after uploading. Based on the method of a previous study, the engagement index (likes plus dislikes plus comments) and the viewing index (views/posting days) were calculated.16 In addition, the presence or absence of personal experience and exercise coaching were recorded. All the videos were categorized into 2 groups according to source and purpose. The classification of the video sources was based on the “About” part on YouTube, which provides self-introduction of the uploader and, if accessible, additional information, including the uploader’s web site, was also referenced. According to videos’ sources, the professional group included videos from official universities and/or hospitals, academic institutions, and independent medical professionals (eg, physicians, physical therapists, and occupational therapists), and the non-professional group included the rest. In terms of videos’ purposes, news, media, personal video or commentary, and advertisements of the program were placed in the non-informational group and the rest in the informational group.
Scoring Systems.
Two independent rehabilitation physicians (M.E.Y and C.W.J), who were blinded to each other and who had >4 clinical years of pulmonary rehabilitation, evaluated all the videos by using the modified DISCERN score and the pulmonary rehabilitation for COPD-specific score (PRSS) after receiving similar training on the evaluation of videos. The content of and information on each video footage were reviewed. When different scores were obtained for the same video, the discrepancy was resolved by a third, independent rehabilitation physician (M.H.B). The modified DISCERN scale is a simplified grading scale with 5 questions designed to evaluate the reliability of the videos.19,20 This tool examines 5 aspects (clarity, reliability, balance/bias, provision of information sources, and mention of uncertainty), which are scored based on a 2-point scale that ranges from 0 to 1. The maximum potential score was 5, with significance in the reliability set at ≥ 3 (Fig. 2A).
A: The modified DISCERN score. B: The pulmonary rehabilitation for COPD-specific score (PRSS).
Due to the unavailability of evaluation tools for assessing the quality of videos related to pulmonary rehabilitation for COPD, a customized scoring system, namely, the PRSS, was developed by us based on 3 pulmonary exercise guidelines from the American College of Sports Medicine, American Thoracic Society/European Respiratory Society, and American Association of Cardiovascular and Pulmonary Rehabilitation.1,21-23 This tool is divided into 9 domains, including the definition, goal, benefit, exercises (endurance, resistance, flexibility, and exercise-others), education, and psychosocial intervention of pulmonary rehabilitation. Each domain represents critical features that are considered important components of pulmonary rehabilitation. Each criterion was scored from 0 to 2, with a possible maximum of 18 points, according to the extent to which a video was mentioned or discussed.
All the videos were assigned a score for each domain, with a score of 0 indicating a domain not addressed or that included misinformation, that of 1 indicating a domain insufficiently addressed, and that of 2 indicating a domain sufficiently addressed in the video. Thereafter, the scores of related domains were combined and used for analysis as follows: PRSS_exercise was the sum of endurance, resistance, flexibility, and exercise-others scores, that is, exercise-related scores; PRSS_intro was the sum of definition, goal, and benefit scores, that is, scores related to basic information on pulmonary rehabilitation; PRSS_others was the sum of education and psychosocial intervention scores, that is, scores of a collection of non-exercise interventions. PRSS_total was the sum of the scores for all the domains. The details of the tool are shown in Figure 2B.
Statistical Analysis
Descriptive data are presented as number (%), mean ± SD, and median (range). The Kolmogorov–Smirnov test was applied to approximate the normality of the data, and the Mann-Whitney U test was used to determine the differences between the groups. The Fisher exact test was performed to investigate the association between the professional and informational groups. The Spearman rank correlation coefficient was used to determine the influence of the variables of the videos on the engagement index and the viewing index. Interrater reliability was measured separately for the scoring of the modified DISCERN and the PRSS scales by using the Cohen weighted kappa coefficient, with significance set at > 0.6. All data were analyzed by using SPSS Statistics (version 20; IBM SPSS, Chicago, Illinois). Statistical significance was set at P < .05.
Results
Descriptive Characteristics of Videos
In total, 100 videos were entered into the database, and, after exclusion, 62 videos were analyzed (Fig. 1). The descriptive characteristics of the videos are presented in Table 1. The median (range) numbers of views and posting days were 3,255 (881–501,970) and 2,081 (239–5,050), respectively. The median (range) viewing index was 3.11 (0.33–200.23). The median numbers of likes, dislikes, and comments were 27 (0–3,300), 1 (0–158), and 0 (0–191), respectively. The median (range) engagement index was 29 (0–3533), and the median (range) duration was 5 (1–50) min.
Summary of Included YouTube Videos (total no. videos = 62)
Categories and Contents of Videos
Most of the videos (54/62 [87%]) were uploaded by professionals and the remainder (8/62 [13%]) by non-professionals. According to their purposes, 35/62 (56%) were informational, and 27/62 (44%) were non-informational (Table 1). Only a few videos (11/62 [18%]) contained a personal experience, and 31/62 (50%) had exercise coaching content.
Reliability and Quality of Videos
The mean ± SD modified DISCERN score for all the videos was 3.76 ± 1.04. The mean ± SD PRSS_total, PRSS_intro, PRSS_exercise, and PRSS-others were 4.13 ± 3.73, 1.21 ± 1.55, 2.13 ± 2.15, and 0.79 ± 0.99, respectively. The mean scores for each domain are presented in Table 1. The distribution by score for each PRSS domain of the videos and ∼50% of the videos did not mention or discuss the contents of all domains, as illustrated Figure 3. The interrater reliability was significant for all scoring systems (0.910 for modified DISCERN and 0.890 for PRSS).
Score distribution for each of the 9 domains of pulmonary rehabilitation for COPD-specific score in all the videos. PI = psychological intervention.
Analysis by Video Categories
The differences in variables between the source and purpose groups are summarized in Tables 2 and 3. When comparing the video groups according to professionalism, the Mann–Whitney U test revealed significant differences only in the modified DISCERN score (P < .001). Results of the Fisher exact test, which examined personal experience, exercise coaching, and informational purposes, did not show any significant differences between the groups. When comparing the informational and non-informational groups, the median engagement index (P = .007), median viewing index (P < .001), mean modified DISCERN (P < .001), mean PRSS_total (P = .001), and mean PRSS_exercise (P < .001) scores were significantly higher in the informational group. The presence of personal experience and exercise coaching was also significantly higher in the informational group (P < .001 and P = .001, respectively).
Comparison of Video Characteristics According to Categorical Distributions
Comparison of Video Characteristics According to Categorical Distributions
Analysis of Engagement Index and Viewing Index
As shown in Table 4, the engagement index was higher when the video contained a personal experience (P = .035). However, the presence of exercise coaching was not a significant factor (P = .16). Conversely, in the case of the viewing index, the presence of personal experience did not have a significant effect (P = .09); nevertheless, exercise coaching was significant (P < .001). The Spearman rank correlation coefficient analysis demonstrated that video duration was positively correlated with the engagement index and viewing index (P = .004 and P = .001, respectively). Among the scores, the modified DISCERN (P = .007) and PRSS_others (P = .02) were significant factors for the viewing index. The other scores exhibited no significant associations.
The Factors Associated with the Engagement Index and the Viewing Index
Discussion
Video resources for patients with COPD have been used in medical research several times.15,24 Liu et al25 reported that a video-based online breathing program for subjects with stable COPD improved pulmonary function, exercise capacity, and health status. One pilot study conducted by Moore et al26 demonstrated that a home exercise video program could improve breathlessness and exercise capacity in subjects with moderate-to-severe COPD. Recently, video materials have been used as a form of telemedicine, and their effectiveness has been proven.8 A study that analyzed YouTube videos for patients with COPD has also been reported.15 However, previous studies evaluated videos for educational content and were based on data from a decade ago. To our knowledge, this is the first study to analyze the content reliability and quality of videos that are likely to be accessed by those who search for information related to pulmonary rehabilitation for COPD on YouTube.
The results of our study showed that the overall video reliability of pulmonary rehabilitation videos for COPD was fair, and the mean modified DISCERN score among videos exceeded 3 points. However, in detail, a large number of the videos yielded no scores in the “provision of information sources” (34%) and “mention of uncertainty” (66%) domains. With regard to the quality of information provided, most of the videos did not score well. Approximately 70% of the videos either failed to provide information across all PRSS domains or provided misinformation for some of them. In addition, 10%–20% of the videos mentioned each domain but did not provide sufficient information. In summary, the pulmonary rehabilitation videos made by reliable producers did not provide informative content at sufficient depth. This finding can be explained by the comparison between the groups. The videos produced by the professional group exhibited significantly higher modified DISCERN scores; however, other scores showed no meaningful differences from those of the non-professional group. Although most videos were created and uploaded by the professional group, these videos were not of good quality. In particular, the professional group did not upload more videos for informational purposes than the non-professional group did.
However, the analysis demonstrated that viewers, perhaps patients with COPD, had greater interest in videos produced for informational purposes. This can be inferred from the finding in which the videos produced for informational purposes obtained significantly higher viewing index and engagement index scores than did those produced for non-informational purposes. Furthermore, videos produced for informational purposes notably obtained significantly higher exercise-related scores among the PRSS domains. In other words, viewers preferred videos produced for informational purposes; nevertheless, such videos were more focused on exercise-related information. However, because pulmonary rehabilitation programs are not of greater importance, the provision of non-exercise–related content should be given further consideration in future video production.
When producing a video, it is important to ensure that many viewers will be interested in it. In this respect, the present study potentially provides new insights. On analyzing the factors that affect viewer engagement in a video, the sharing of patient experiences and the video length were significant. An analysis of viewership-related factors revealed that video reliability and length had an impact. It should be noted that the longer the video, the more interested the viewers were. Although the videos with play times of >1 h were not included in the study, we considered viewers to prefer long-duration videos that provide sufficient information to short-duration videos that lack information.
The strength of this study is in its use of a novel scoring system, namely, the PRSS. Previous studies evaluated the quality of YouTube videos by using the adapted clinical guidelines related to the health condition or customized tools developed by us.27-29 In particular, in the field of respiratory diseases, YouTube videos for educational contents related to idiopathic pulmonary fibrosis and asthma were evaluated by newly developed scoring systems.14,17 However, to our knowledge, there has been no tool yet to evaluate the quality of YouTube videos for pulmonary rehabilitation for COPD. The PRSS is the first tool to evaluate the contents of pulmonary rehabilitation in videos. This scoring system has been developed for patients with COPD but could later be adapted to evaluate the quality of videos related to rehabilitation of other respiratory diseases based on the 3 guidelines that are the basis of the PRSS.1,21-23
Limitations
There are certain limitations to our study. First, we used only one keyword, “pulmonary rehabilitation,” in this study. If other similar key words, such as “PR” or “respiratory rehabilitation” were used, the search results might have changed. However, we confirmed through Google Trend (http://trends.google.com) that the term “pulmonary rehabilitation” is most commonly used by users who want to find online information with regard to pulmonary rehabilitation. Thus, we considered this term the best term for the search. Second, our study was limited to the 100 most viewed videos, exclusively in English; thus, videos outside these results were excluded. The sensitivity could have increased if more videos in other languages were included. Third, our results exclusively represent the state of publicly available YouTube videos on the search date and time. Given the dynamic characteristics of YouTube, in which many videos are uploaded and deleted in real time, findings can change over time. Fourth, when long content was divided into multiple videos, each video was analyzed as a separate video. These videos would have received low scores on reliability and quality.
Also, several guidelines and recommendations, not used in this study, are available, and the content of pulmonary rehabilitation programs varies globally. In addition, the prescribed program may vary, depending on the patient’s disease variability and severity. To overcome this, we developed a new evaluation tool, the PRSS, to assess adherence to the most basic content based on the guidelines created by 4 leading professional societies.1,21-23 We felt that, with the high interrater reliability agreement, it is likely a reasonably useful tool, given the absence of a standardized alternative. However, the PRSS is not a validated assessment tool, and we agree that future research is needed to establish its validity.
Conclusions
The results of this study indicated that popular YouTube videos on pulmonary rehabilitation for COPD were reliable; however, they were biased and of low quality in terms of providing content. In addition, although videos produced for informational purposes were of relatively high quality and interest to viewers, the professionals who posted most of these videos have been reluctant to upload informational videos. Given YouTube’s advantage of easy accessibility and its disadvantage of vulnerability to poor-quality information, professionals should pay more attention to uploading comprehensive high-quality videos for informational purposes, taking into account factors that can increase viewer participation. These findings may be used by professionals who intend to upload videos on this topic in the future.
Acknowledgments
We thank Editage (www.editage.co.kr) for English language editing.
Footnotes
- Correspondence: Chan Woong Jang MD, Department of Rehabilitation Medicine, Gangnam Severance Hospital, Rehabilitation Institute of Neuromuscular Disease, Yonsei University College of Medicine, 211 Eonju-ro, Gangnam-gu, Seoul 06273, Korea. E-mail: dearjcw89{at}gmail.com
The authors have disclosed no conflicts of interest.
- Copyright © 2022 by Daedalus Enterprises
