Abstract
BACKGROUND: The 6-min walk test (6MWT) is an important tool in the assessment of functional capacity and prognosis in patients with COPD. However, especially in long-term follow-up in clinical settings, this test may be executed by a different assessor, and it is not well known whether 6MWT has an acceptable inter-rater reliability. The aim of this study is to analyze the intra- and inter-rater reliability of the performance in 6MWT, its cardiorespiratory changes, and effort perception in subjects with COPD.
METHODS: Thirty-two subjects with a diagnosis of COPD participated in the study, but 3 subjects did not appear on the second day of evaluation and therefore were included only in the intra-rater analysis; the first and second tests were executed by the same assessor with a 30-min interval between them, and the last was executed by a different assessor a week later. The intra-rater reliability was verified comparing the first and second 6MWT performance, and the inter-rater reliability was verified comparing the third test with the best performance of the first and second tests.
RESULTS: The intraclass correlation coefficient values were >0.75 (P < .001) for the walked distance on the 6MWT; however, the limits of agreement, SE of measurement, and minimal detectable difference were higher than the minimum clinically important differences already mentioned in the literature (∼25, 26, and 54 m), and the coefficient of variation was small in both intra- and inter-rater comparisons.
CONCLUSIONS: The 6MWT showed excellent reliability for distance and perceived exertion and moderate to excellent for HR and SD as assessed by intra- and inter-rater analysis. Thus, based on the main study outcomes, we concluded that the 6MWT can be compared when conducted by 2 different evaluators.
Introduction
In clinical practice, functional tests are important tools to assess functional capacity of subjects with COPD.1 The accepted standard test to identify the cause of exercise intolerance is the symptom-limited cardiopulmonary exercise testing with direct measurements of exhaled gases.1,2 However, this test is complex and expensive and requires specialized staff, precluding its use in clinical routine.3 As an alternative to cardiopulmonary exercise testing, the 6-min walk test (6MWT) has been commonly used, since its cost is low, it is easy to perform, and it has been used to assess functional capacity before and after interventions.4
Currently, the 6MWT is well standardized by the American Thoracic Society/European Respiratory Society (ATS/ERS)4 and is the most often used test in clinical practice to determine the functional capacity of patients with chronic lung disease. Moreover, 6MWT is also a predictor of morbidity and mortality5–7 and may reflect the ability to perform activities of daily living in patients with COPD.8
Furthermore, 6MWT presents an essential role in the evaluation of functional capacity, once it provides a global and integrated response of both physical (pulmonary and nonpulmonary) factors and subjective effort perception. In addition, the 6MWT can verify physiological adaptation during physical stress.8,9 This test has been considered an important tool in rehabilitation programs designed for patients with COPD.10 Additionally, the study of the reliability of the 6MWT is important for a better interpretation of its results.
6MWT is reliable in subjects with COPD,4,11–16 but a learning effect occurs, since subjects reach a considerably greater distance when performing a second test. In fact, there is controversy about the size of the learning effect, which can vary from 2.6 to 22%.12,14,17–21
Moreover, in some situations, such as a long-term follow-up in clinical practice, the 6MWT may be conducted by different assessors, causing an error rate that could complicate the interpretation of changes in the functional capacity. Comparisons of tests conducted by different assessors should be made with caution, since an increase in the error is to be expected due to interpersonal differences in voice intonation, verbal incentives, the ability to assess the changes in physiological measures, and trust of the subject in the assessor. Therefore, the use of tests conducted by different assessors may increase the error rate, which could complicate the interpretation of changes in the functional capacity. However, few studies22,23 have tested the inter-rater reliability of the 6MWT, and they assessed only a population with multiple sclerosis. Therefore, the inter-rater reliability of the 6MWT is still unknown in subjects with COPD, which explains the relevance of our study, and the improvement of cardiorespiratory changes and perceived exertion during the 6MWT have been little explored. It is therefore necessary for further studies to be carried out, considering the importance of the 6MWT in clinical practice in subjects with COPD.
The aim of this study is to analyze the intra- and inter-rater reliability of the performance in 6MWT, its cardiorespiratory changes, and effort perception in subjects with COPD. The authors hypothesized that 6MWT would be reliable in inter-rater analysis but would not be reliable in intra-rater analysis due to the learning effect between the first 2 tests.
QUICK LOOK
Current knowledge
There are already several studies in the literature showing the intra-rater reliability of the distance walked in the 6-min walk test (6MWT) in populations with several chronic diseases, especially in patients with COPD.
What this paper contributes to our knowledge
Our paper analyzes of inter-rater reliability for the distance walked during the 6MWT and changes in cardiorespiratory variables and perceived exertion in subjects with COPD. There are no studies in subjects with COPD who underwent 2 6MWTs conducted by 2 different evaluators.
Methods
Study Design and Participants
This study is a part of a larger a prospective, cross-sectional observational study from the Spirometry and Respiratory Physiotherapy Laboratory at the Federal University of São Carlos, Brazil. This larger study intended to verify the psychometric properties of the 6-min step test and the 6MWT in 3 different populations (subjects with COPD, young adults, and older adults) and was registered in ClinicalTrials.gov (identifier NCT01298661). This paper was a prospective cross-sectional study, which focused upon the reliability of 6MWT in subjects with COPD.
The enrollment period was from January 2011 to June 2012, and subjects were invited using posters in the university and its neighborhood, on local radio and television, and in the newspaper. Furthermore, patients referred for treatment to the Unit of Respiratory Physiotherapy-Federal University of São Carlos were also invited to participate. All subjects willing to participate were asked during a telephone call whether they matched the inclusion criteria to participate in the study. The following inclusion criteria were adopted: subjects with COPD diagnosis24 who did not show other diseases that might preclude them from performing the proposed tests. Subjects were invited for an initial assessment when they signed the consent form and were asked whether they had already performed a 6MWT before this study or presented an exacerbation in the 1 month before the study; in cases where the subject responded affirmatively to one of those questions, the subject was excluded from the study. Subjects who could not complete the first assessment day were not included in the analysis, and those who did not attend the second assessment day were not included in the inter-rater reliability analysis.
The study was approved by the university human ethics committee (decision number 009/2011). The consent and the ethics committee-approved protocol included all of the objectives and measurements carried out in this paper.
Experimental Procedures
Subjects' assessment was conducted in 2 days, with a 1-week interval. On the first day, the subjects' history and their anthropometric characteristics were collected in an initial assessment. On the same day, they performed 2 6MWTs, with a 30-min interval between them, conducted by assessor 1. On the second day they underwent a body composition analysis and a third 6MWT conducted by assessor 2.
Anthropometric and Body Composition Analysis
Subjects were asked to remove their shoes and to wear light clothes during the measurement of weight and height (biometric scale model 110FF, Welmy, São Paulo, Brazil). In addition, body mass index was calculated: body mass index = weight (kg)/height (m)2. Using a body composition analyzer (model BC-553, Tanita, Arlington Heights, Illinois), fat-free mass and fat mass as a percentage of the total weight (fat percentage) were obtained, and the fat-free mass index was calculated: fat-free mass index = fat-free mass (kg)/height (m)2.25 For this analysis, subjects were instructed not to eat or drink anything for 4 h before the measurement.
Spirometry
A pre- and post-bronchodilator assessment was conducted previously by the pulmonologist to verify the degree of obstruction, according to guidelines of the ATS/ERS,26 and the obtained values were FVC, FEV1, and FEV1/FVC. FEV1 and FVC were expressed as a percentage of the predicted values, using Brazilian reference values.27
6-Min Walk Test
The 6MWT was performed twice, with a 30-min interval, in a corridor of 30 m, with 3-m demarcations, according to ATS/ERS recommendations.4 Subjects were asked to walk as far as they could during 6 min, receiving standardized verbal incentives each minute.
In the beginning and in the end of the test, pulse oximetry (model 2500 oximeter, Nonin Medical, Plymouth, Minnesota), heart rate (Vantage NV, model 1901001, Polar, Kempele, Oulu, Finland), blood pressure and effort perception using a modified Borg scale 0-10 were verified.28 Performance in this test was expressed using the distance walked (6MWD) in meters.
Statistical Analysis
Data distribution was verified by a Shapiro-Wilk test and was expressed using mean ± SD. P values <.05 were considered as significant. To verify the intra-rater reliability, the first and second 6MWT were compared, and the inter-rater reliability was verified by comparison of the first, second, and the best performance in the first or second test with the third test.
The intraclass correlation coefficient (ICC) was used to verify reliability. Reliability was considered to be excellent when the ICC was >0.75.29 Moreover, a one-way repeated measures analysis of variance test was used to compare the means of the first, second, and third 6MWT. The coefficient of variation (SD/mean),30 the SE of measurement (SD × √[1 − ICC]), and the minimum detectable difference (1.64 × √2 × SE measurement)31 were calculated. Furthermore, the mean error and limits of agreement were displayed in Bland-Altman plots.
The required sample size to perform the proposed reliability analysis was at least 19 and 13 subjects,32 considering α <.05 and beta <0.2, a number of test repetitions of 2 and 3 respectively, the null hypothesis of ICC <0.7, and the expected hypothesis of ICC>0.9. The expected ICC value used was consistent with the mean value of ICC found in 6MWT reliability studies, which is 0.82–0.997,12,14,15,33–35
Results
This study included 34 subjects, and 2 of them were excluded because they did not complete the first assessment. Three subjects did not appear on the second day of evaluation, and they were included only in the intra-rater analysis.
Sample characteristics are shown in Table 1. On average, subjects walked 370.7 m (95% CI 278.4–662.6) in the first 6MWT, 386.6 m (95% CI 281.8–717.9) in the second 6MWT, 394.5 m (95% CI 291.1–721.4) in the best 6MWT of the first and second, and 392.4 m (95% CI 284.9–732.5) in the third 6MWT. The distance walked in the second test increased on average by 15.9 m (95% CI 13.4–23.7 m), distance in the third with respect to the first 6MWT increased on average 21.7 m (95% CI 19.3–24.1 m), and distance in the third in the respect to the second 6MWT increased on average 5.8 m (95% CI 3.4–8.2 m). There were no statistically significant differences between the values of T1, T2, T1 or T2, and T3 for the 6MWT, which corroborates with the reliability of the 6MWT (Table 1).
Characteristics and Exercise Capacity of the Subjects
The ICC values were >0.75 (P < .001) for the walked distance during the 6MWT. However, error analysis, verified by the Bland-Altman plot, was considered high for all analyses; the SE of measurement and the minimum detectable difference showed high error; and the coefficient of variation was small in both intra-rater and inter-rater comparisons (Table 2 and Fig. 1); and were presented ICC values, coefficient of variation, mean error, and limits of agreement for cardiorespiratory changes and effort perception (Table 3).
Reliability of the 6-Min Walk Test
Bland-Altman plot for absolute reproducibility of the 6-min walk test (6MWT) in subjects with COPD. A: Intra-rater assessment for first and second 6-min walk test. B: Inter-rater assessment for the third 6MWT and best 6MWT (best performance among the first and second tests). C: Inter-rater assessment for the third and first 6MWT. D: Inter-rater assessment for the third and second 6MWT. Solid lines indicate the mean error, and dashed lines denote the limits of agreement.
Reliability of the Cardiorespiratory Changes and Effort Perception in the 6MWT
Discussion
Several studies2,7,15,16 have analyzed the intra-rater reliability of the 6MWT; therefore, this test has been considered reliable for assessing functional capacity in patients with COPD after a practice test. However, there is a lack of studies verifying the inter-rater reliability for this population.
The intra-rater 6MWT reliability in our study presented ICC values for walked distance >0.75, indicating excellent reliability. This analysis has been already studied in subjects with chronic respiratory disease by many authors, who found ICC values ranging from 0.82 to 0.99,7,12,14,15,33–35 confirming the findings of our study. The studies mentioned above were conducted with COPD,7,15,34 with obstructive disease and restrictive lung diseases,12 and with lung disease in the final stage.35 The last 2 studies not performed the second 6MWT, with an interval of 30 min after the first 6MWT, according to the standards of the ATS/ERS.7,14 Furthermore, we found low coefficient of variation values (0.06), which indicate that our data are homogeneous. One study verified the reliability of the 6MWT in subjects with cystic fibrosis, and 2 studies verified its reliability in subjects with COPD11,36,37; they also found coefficient of variation values similar to those found in our study (0.04–0.05).
Nevertheless, the error values, which are an important way to determine the reliability, were considered high in the intra-rater analysis of the 6MWT, since the minimum detectable difference was higher than the minimum clinically important differences, ∼25 m,38 26 m,39 and 54 m.40 When evaluated by SE of measurement, the 6MWT error values found by other authors were lower than the one found in this study, ranging from 15 to 20 m.17,34 Hernandes et al7 found a mean error of 27 m with limits of agreement (Bland-Altman analysis) of −60 to 120 m being the amplitude between these values (187 m), very similar to what was found in the present study (185 m); however, the mean error of the present study was <13 m, and the tests were not performed on the same day.
Another study14 found a mean error of 20.1 m with limits of agreement of −60 to 107 m, similar to what was found in this study; however, it should be noted that the tests were not performed on the same day, which differs from the method used in this study and the recommendations. Roomi et al41 found a mean error of 0.65 m and limits of agreement of −62 to 64 m, but the second test was conducted between 2 and 10 d after the completion of the first test, and the tests were not driven by the same assessor. The high error in the intra-rater analysis suggests that probably a learning effect14 occurs, which corroborates the idea that it is necessary to perform at least one practice test to lessen this effect, as recommended by the ATS/ERS.4
Regarding the comparison by analysis of variance, no significant differences were found between any of the ways of analyzing 6MWT in this study, in disagreement with other studies,7,11,12,17,20,33 but Andersson et al34 found an improvement in the walked distance of 22 m between tests, which was similar to our findings (improvement of 16 m), and the authors stated that this improvement may be explained by the learning effect and/or due to the motivational aspects of subjects.
As presented above, several studies have studied the intra-rater reliability of the 6MWT performance. However, the inter-rater reliability was only verified by 2 studies22,23 in subjects with multiple sclerosis, and they presented ICC values ranging from 0.96 to 0.99, and when evaluated by SE of measurement, the 6MWT error found by them was 35.85 m22 and 24.76 m,23 similar to the findings of our study. However, the study by Toomey and Coote23 presented a mean error of 5.22 m with limits of agreement of −18.30 to 28.74 m. This is lower than in our study, but the interval between tests was not described in the study, and the authors reported that one of the study's limitations was a small sample size, smaller than in the present study. Furthermore, we found low values of the coefficient of variation (0.07) for inter-rater reproducibility for 6MWT performance, which indicates that our results were homogeneous, but we did not find any other study that conducted this type of analysis between different evaluators.
Regarding the reliability of the cardiorespiratory changes during the 6MWT, heart rate change in the 6MWT in the intra-rater and inter-rater analysis presented excellent (ICC = 0.80) and poor to moderate (ICC = 0.55) reliability. However, Hernandes et al7 found that the intra-rater reliability was only moderate (ICC = 0.62), and although the sample size was larger than ours, the 2 6MWTs were performed on different days, which may explain the differences between the results. The coefficient of variation value found in our study was moderate (0.14), indicating good homogeneity, since the coefficient of variation values in 2 studies (with cystic fibrosis and other COPD)36,37 showed low values ranging from 0.04 to 0.07. However, in both studies, the first and second 6MWD were performed on the same day, but with a different time interval between tests. As for inter-rater reliability, we found low values of coefficient of variation for change in heart rate (0.07), but no other study evaluated the reproducibility between different evaluators.
The change in pulse oxygen saturation during the 6MWT was not reliable in intra-rater (ICC = 0.16) and inter-rater (ICC = 0.15) analysis in our study. In addition, the study by Chatterjee et al16 showed that the reliability for oxygen saturation in the 6MWT was only modest to determine the need for ambulatory oxygen (SpO2 ≤ 88% for ≥5 s) in stable subjects with COPD actively participating in a pulmonary rehabilitation program, when 3 6MWTs were performed (kappa statistic = 0.62, 72% agreement between measurements).
Moreover, for analysis of intra-rater reliability, the value of the coefficient of variation that we found for the change in pulse oxygen saturation was very high (−0.88), similar to the finding of a study conducted in subjects with interstitial lung disease,42 which also showed a high value of coefficient of variation (0.28). Hernandes et al7 found a change in the pulse oxygen saturation limits of agreement ranging from −7 to 8%, similar to our study (−8.8 to 9.2%); however, the study in question presents a large sample, but the first and second 6MWT were performed on different days. Also, there have been found values very high of the coefficient of variation for pulse oxygen saturation (−0.95) for inter-rater reliability, but we did not find any study that conducted analysis by different evaluators.
As for the change in sensation of dyspnea (ICC in intra-rater analysis = 0.73 and in inter-rater analysis = 0.53) and lower limb fatigue (ICC in intra-rater analysis = 0.82 and inter-rater analysis = 0.84), reliability was moderate and excellent, respectively. However, the study by Hernandes et al7 found only modest reliability (ICC = 0.59 for both), probably due to the fact that the first and second 6MWT were performed on different days. Our study found a moderate coefficient of variation value for change in the sensation of dyspnea (0.29) in the intra-rater reliability, similar to a study with COPD37 (0.22). To analyze inter-rater reliability, the coefficient of variation values for sensation of dyspnea (0.36) and lower limb fatigue (0.36) were very high, and we did not find any study that made an evaluation between different evaluators. Thus, the clinical implication of our results is evident, showing that the walked distance and perceived exertion during the 6MWT are reproducible when conducted by different evaluators; therefore, we believe that the tests can be compared.
The strength of our study was to evaluate the reliability of the distance walked during the 6MWT in subjects with COPD and change variables in cardiorespiratory changes and effort perception between different evaluators. This study presents a limitation in that it used a smaller sample than other studies for the same purpose. In addition, because a third test with the same evaluator was not performed, it was not possible to identify whether the error rates found would be lower when interference from the learning effect or interpersonal variability of evaluators was removed.
Conclusions
The 6MWT showed excellent reliability for distance and perceived exertion in both intra- and inter-rater analysis. For cardiorespiratory variables, reliability was moderate to excellent as assessed by intra- and inter-rater analysis, except for pulse oxygen saturation. Thus, based on the main study outcomes, we concluded that the 6MWT can be compared when conducted by 2 different evaluators.
Acknowledgments
The authors thank all who participated as subjects in this study and in a way contributed to its completion.
Footnotes
- Correspondence: Ivana Gonçalves Labadessa, Spirometry and Respiratory Physiotherapy Laboratory, Department of Physiotherapy, Universidade Federal de São Carlos (UFSCar), Rod, Washington Luis, Km 235, Caixa Postal 696, CEP 13565-905, São Carlos, SP, Brazil. E-mail: ivanaresp{at}ufscar.br.
This work was supported by scholarships from the Conselho Nacional de Pesquisa e Desenvolvimento Científico e Tecnológico (CNPq) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).
The authors have disclosed no conflicts of interest.
Dr Arcuri presented a version of this paper at the 23rd European Respiratory Society Annual Congress 2013, held September 7–11, 2013, in Barcelona, Spain.
- Copyright © 2016 by Daedalus Enterprises
