Linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care

Outline of problem

In our surgical intensive care unit, 46 000 arterial blood gas analyses were performed each year. A one week prospective study showed that over half of these tests could not be justified clinically. In addition, 96% of requests were left to the discretion of the nursing staff, while clinical signs such as respiratory rate or altered pattern of breathing were seldom taken into account in deciding whether the test was necessary. Values of percutaneous oxygen saturation from pulse oximetry were rarely used, even though they match arterial measurements. These findings led us to set up an intervention aimed at reducing the number of requests.

Outline of context

The intervention was conducted in the surgical intensive care unit of a 1200 bed tertiary, university affiliated hospital. The 20 bed unit receives 1500 patients per year for postoperative care after major surgery, multiple trauma, neurosurgery, and organ transplantation for a total of 5300 hospital days. The team comprises three senior intensive care specialists, 10 physicians in training, and 71 nurses (69% certified in intensive care nursing, 15% in second year training, and 15% in first year training). Blood gas analysis is performed at the patient's bedside with three Stat Profile Ultra machines (NOVA biomedical, Waltham, MA) located in the unit. All patients are equipped with a pulse oximeter for monitoring of percutaneous oxygen saturation. Capnography is not used in the unit because it is not reliable in critically ill patients.8 Arterial catheters are inserted in most patients as clinically required.

Detail of approach

Guidelines and feedback have been proposed in the literature as a means of changing ordering habits for routine blood gas analysis. A guideline specifies the rules to apply when performing analyses and helps the user to make an adequate decision.9-11 Feedback, either concurrent 12 13 or retrospective,14 allows users to correct their behaviour. It can be provided on a regular basis 12 13 or triggered by thresholds.14 It can focus on adherence to rules13 or on impact on volumes14 or costs12 of laboratory tests.

To improve the use of blood gas analysis, we designed a multifaceted intervention15 that combined a new guideline, educational sessions, and feedback. To achieve consensus of all involved parties and facilitate acceptance of the guideline, this intervention was carried out by a multidisciplinary group.16

Guideline development

A pilot version of the guideline was designed locally by a group comprising one intensive care consultant, one intensive care senior registrar, three nurses (certified and in training), and one respiratory therapist. From October to December 1997, the group met weekly to define the decision making process underlying the measurement of three blood gas parameters (pH, PaO₂, PaCO₂). The guideline was developed on a consensus basis17 as an extensive literature search in the Cochrane Library and Medline did not identify any suitable guideline.

Fig 1

Algorithm for requests for arterial blood gas analysis (translated from French)

• Download figure
• Open in new tab
• Download powerpoint

The pilot guideline, which included a comprehensive time frame and the use of pulse oximetry, was devised as an algorithm based on three pathways corresponding to pH, PaO₂, and PaCO₂ and included a minimum number of three mandatory analyses per day for safety reasons. The guideline was approved by the two other unit consultants.

To incorporate the opinion and the experience of the users we amended the pilot version of the guideline ten months later. In this consolidated version, the time frames were widened and the daily mandatory tests removed. We added a list of clinical settings in which the algorithm should not be applied. The consolidated guideline was validated by four critical care experts from outside the unit (fig 1).

Intervention

Both pilot and consolidated guidelines were distributed as pocket-sized documents and were displayed next to blood gas analysers. They were introduced to all team members in the surgical intensive care unit during 45 minute teaching sessions twice a week over one month. In addition, all new team members were individually instructed when they arrived in the unit.

Feedback was given to the whole team by using two indicators. The adherence indicator revealed deviations from the guideline and the blood gas indicator showed the impact of increased adherence, defined here as appropriateness,11 on the use of blood gas analysis. Both indicators were presented systematically each month, plotted as time series charts, displayed on walls, and published in the unit information bulletin.

Measurement of problem

Intervention assessment and data collection

We compared three consecutive ten month periods (baseline, pilot, consolidated). They were separated by the dissemination of the pilot version and the consolidated version of the guideline, respectively. Data were obtained retrospectively for the baseline period and prospectively for the pilot and consolidation periods and tabulated monthly.

A first set of data was obtained by sampling one day a month (the audit day). For all patients who were in the unit on that day, two investigators independent of the care team collected data on age, the simplified acute physiology score, version 2 (SAPS II)18 as a measurement of the severity of illness, and the type of surgery. They individually assessed the proportion of blood gas analyses that complied with the guideline (adherence). All results were checked between the two observers and each disagreement was discussed to achieve consensus. As undetected severe acidaemia or hypoxaemia might lead to major cardiac arrhythmia, cardiac or respiratory arrests, new episodes of pH <7.25, or systolic systemic blood pressure <80 mm Hg, any such incidents were recorded by the attending nurse on the electronic chart. Two independent reviewers computed the number of incidents per day (incident rate) at the end of the patient's stay in the unit to assess the safety of the implemented procedure. Results were compared between the two reviewers, and all discrepancies were double checked and corrected. For patients who stayed long enough to be involved in two consecutive audits, the incident rate from one audit was used.

A second set of data consisted of aggregates extracted from the hospital information system for the whole unit. We tabulated the monthly average number of blood gas analyses per patient day, the average number of all other tests performed per patient day (other tests), and the average length of stay. In addition, we extracted from the hospital information system, the mortality for the whole unit population for the three periods. For a given month, the average adherence observed during the corresponding audit day served as adherence indicator, whereas number of tests per patient day was used as the arterial blood gas indicator.

Fig 2

Monthly change in absolute number of arterial blood gas analyses per patient day and proportion of tests adhering to current guideline from baseline (before implementation of pilot guideline), pilot period (after introduction of pilot guideline), and consolidation (after implementation of consolidated guideline)

• Download figure
• Open in new tab
• Download powerpoint

Cost-benefit analysis

We calculated benefits associated with increased appropriateness of requests in terms of amount of blood taken (5 ml per test), nurse working days (4 minutes per test), number of analyses per patient day, and corresponding costs (SFr 20 (£8) per test). We assessed the cost of the whole intervention, including meetings, teaching sessions, and reviews, in terms of collaborators' time (number of hours).

Fig 3

Number of arterial blood gas analyses per patient day per 10 month period (median with minimum and maximum values; 25th and 75th centiles indicated by box plot). P<0.001 for comparison of three periods (Kruskal-Wallis)

• Download figure
• Open in new tab
• Download powerpoint

Statistical analysis

To perform statistical comparisons, we grouped monthly data together within each period. All variables were compared by the Kruskal-Wallis test except for adherence, mortality, and type of surgery, for which we used Pearson's χ² test. Data analyses were performed with SPSS release 9.0. P <0.05 was considered significant.

This study was approved by our institution's ethical committee for human research, who authorised a waiving of informed consent. The whole programme was followed and supervised by the Hospital Quality of Care Unit.

Feedback

Figure 2 shows the time series chart reported monthly to the whole team. It shows the regular decrease of the blood gas analysis indicator and the concomitant increase of the adherence indicator after the introduction of the pilot and subsequently the consolidated guideline.

Impact of the intervention

During the three periods, the characteristics of the patient population studied remained unchanged with regard to age, acute physiology score, and type of surgery (table). Between baseline and pilot periods, the average number of blood gas analyses decreased from 8.2 to 6.5 per patient day (fig 3), whereas the average adherence rose from 53% to 68% (fig 4). Between pilot and consolidation periods, the average number of tests further decreased from 6.5 to 4.8 (fig 3) and the average adherence rose from 68% to 80% (fig 4).

The numbers of all other tests performed in the unit did not vary significantly during pilot and consolidation periods (mean (SD) 204, 242, and 222 for the three periods respectively, P=0.05). The incident rate, length of stay, and mortality remained unchanged throughout the study (table).

Fig 4

Adherence to guideline per 10 month period (percentage with 95% confidence intervals). P<0.0001 for comparison periods (Pearson's χ²)

• Download figure
• Open in new tab
• Download powerpoint

Cost-benefit analysis

Per patient day, this reduction resulted in savings of 8.5 ml and 17 ml of patients' blood, 6.8 and 13.6 minutes of nurse working time, and SFr 34.8 (£14.15) and SFr 68.4 (£27.81) in test costs for the two periods, respectively. The global reduction in requests for blood gas analysis during the 20 month study therefore resulted in savings equivalent to 112 litres of patients' blood, 187 nurse working days, and SFr 452 600 (£180 369).

The total cost of the intervention was SFr 79 050 (£31 502) (or SFr 9 (£3.60) per patient day), representing 710 hours for nurses, 270 hours for respiratory therapists, and 400 hours for physicians. Follow up of the process—that is, monthly collection and report of the number of tests—can be carried out by one person at negligible cost (four hours a month).

Lessons learnt

Human factor specialists have shown that deficiencies of attention or reasoning are common and that even experienced and responsible professionals can err. To prevent errors, the use of an explicit model such as a guideline is essential in complex decisions.19 However, guidelines can be rejected because of their content or format, especially if users are not involved in their design. Allowing users to participate in the revision of this guideline facilitated its acceptance in our project. By providing monthly feedback, both on use of blood gas analysis and on appropriateness through multiple sources over 20 months, we stimulated learning and education in users. In particular, we were able to show that when the guideline was applied, it could result in a safe reduction in the number of requests. As proposed by others,15 our results favour the combination of multiple approaches in attempts to change professional practice.

The goal of quality improvement projects is not to show without any bias that a specific pinpoint intervention is instrumental in changing practice, especially when already convincing methods are available.15 It aims much more at making this change happen. This requires a managerial approach, combining strategies and dealing with local resources and barriers. We therefore opted for a long term approach that facilitated the implementation of our guideline, using periodic measurements and feedback. As a consequence, less attention was paid to strict control of any bias. For instance, we chose a before and after study design, as well as retrospective data for establishing baseline, and we did not examine the question of inter-rater reliability. However, the decrease in the use of blood gas analysis can hardly be attributed to external independent factors. Indeed, patient characteristics remained stable and the number of all other tests performed in our intensive care unit did not vary.

Next steps

To maintain the benefits of our intervention, a permanent follow up of this project should be engaged. Firstly, we will set up a periodic revision of our guideline. Indeed, despite satisfying the requirements regarding the methodological standards on guideline development and format recently published in two major medical journals, 20 21 our guideline was based on a consensus of experts. Therefore it could benefit from regular reviews of the literature, and its rating on the scale of evidence could be progressively increased. Secondly, we will replace the time consuming monthly audit day with continuous low cost monitoring capable of detecting a decrease in appropriateness of requests. This could be done by using the sole arterial blood gas analysis indicator, which can be easily extracted from our hospital computer system. This indicator could be plotted as a control chart22 and trigger a measure of appropriateness based on a review of patients' records each time it exceeds a statistically predefined threshold. Coupled to a measure of appropriateness every six months, we should be able to control deviations permanently.