Procedure to Normalize Data for Benchmarking

INTRODUCTION: The hospital billing system is usually the source for reporting activity counts used in benchmarking efforts. Because billing is associated with a specific procedure, benchmarking data are often reported as procedure-days, procedure-shifts, or procedure-hours. Normalizing (usually to procedure-days) is required when comparing data for benchmarking purposes. For an institution that uses hourly billing, simply dividing procedure-hours by 24 (or procedure-shifts by 2 or 3) will underestimate the procedure-days reported by a daily billing system, because daily billing systems use the convention that any fractional day of service is rounded up to the next higher day. The purposes of this study were: (1) to simulate sets of data and determine the expected error with conversion by simple division, (2) to derive a more accurate procedure for normalizing benchmarking data, and (3) to compare the new normalization procedure to simple division, using simulated and actual data.

METHODS: A reference population of simulated patient data was created using a spreadsheet to generate random start times paired with actual procedure durations (eg, hours of mechanical ventilation) for 5,000 patients. The spreadsheet calculated “true” billable procedure-days and procedure-shifts from the simulated procedure-hours. Next, a resampling procedure was used to simulate the effect of submitting benchmarking data based on various numbers of patients. The resulting sets of data were used to examine the association between sample size and conversion error when converting from procedure-hours to procedure-days and to generate an alternative conversion procedure that uses linear regression to estimate procedure-days from procedure-hours. An additional regression equation was generated from actual patient data, using simultaneously recorded procedure-hours and proceduredays. The set of mean conversion errors for the 2 regression equations was compared using the MannWhitney rank sum test.

RESULTS: In general, conversion errors (both systematic and random errors) were smaller with larger sample sizes and with longer service periods, approaching an asymptote at a sample size greater than about 20. Using division, the conversion errors for a sample size of 100 were −16% for hourly reporting, −11% for 8-hour shifts, and −8% for 12-hour shifts. The regression equations for conversion derived from simulated data were as follows. For hourly billing, procedure-days = −0.237 + (0.049) (procedure-hours). For 8-hour shifts, procedure-days = −0.205 + (0.372) (procedure-shifts). For 12-hour shifts, procedure-days = −0.114 + (0.541) (procedure-shifts). Using those regression equations, the conversion errors for a sample size of 100 were 1% for hourly reporting, −0.2% for 8-hour shifts, and −0.2% for 12-hour shifts. The regression equation (for hourly billing) derived from simulated data gave better results than did the equation derived from actual data (median error 0.39 vs −2.92, p = 0.013).