The aim of this study was to evaluate the short-term benefits of a pulmonary rehabilitation program in chronic obstructive pulmonary disease (COPD) patients. The study was a randomized controlled trial that included 54 mild and moderate COPD patients. Patients were assigned to either an 8-week-long pulmonary rehabilitation program, which consisted of exercise plus education (rehabilitation group), or were controls. All the patients were evaluated at baseline at the completion of the 8th week of the program and one month after the completion of the pulmonary rehabilitation program using five instruments: arterial blood gas analysis, postbronchodilator pulmonary function test, 6-minute walk test (6MWT), Saint George Respiratory Questionnaire (SGRQ), and the dyspnea visual analog scale (VAS) There were no statistically significant differences in the pulmonary functions and pulmonary gas analysis between baseline, discharge (8th week), and the 12th-week visit in both groups (p > 0.05). Rehabilitation resulted in significant improvements in both the VAS and the 6MWT at the 8th week, but by the 12th week all of these improvements had deteriorated. All of the SGRQ domains improved both at the 8th and the 12th week, with a significant difference between the groups (p < 0.05). We conclude that rehabilitation resulted in improvements in exercise capacity, health status, and dyspnea. All of these benefits, however, tend to deteriorate in the first month after rehabilitation. Therefore, it is strongly recommended that all patients with COPD be kept motivated in order to continue with rehabilitation and maintain the benefits gained.