Intended for healthcare professionals

Papers

Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals

BMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38413.576713.AE (Published 05 May 2005) Cite this as: BMJ 2005;330:1057
  1. Catherine Hewitt, PhD student (ceh121{at}york.ac.uk)1,
  2. Seokyung Hahn, assistant professor2,
  3. David J Torgerson, director1,
  4. Judith Watson, research fellow1,
  5. J Martin Bland, professor of health statistics1
  1. 1 York Trials Unit, Department of Health Sciences, University of York, York YO10 5DD,
  2. 2 Medical Research Collaborating Center, Seoul National University Hospital/College of Medicine, Seoul 110-744, Korea
  1. Correspondence to: C Hewitt
  • Accepted 28 January 2005

Introduction

In randomised controlled trials, allocation concealment (separating the process of randomisation from the recruitment of participants) is important for rigorously designed trials.14 In 1996 many major medical journals adopted the CONSORT statement (whereby researchers have to include a short checklist of essential items and a flow diagram when reporting trials),5 and this move encouraged the reporting of allocation concealment. We reviewed the prevalence of adequate allocation concealment and its association with the statistical significance of trial results.

Methods and results

We searched by hand four general medical journals (the BMJ, JAMA, the Lancet, and the New England Journal of Medicine) to identify randomised controlled trials published from January 2002 to December 2002. We included articles if the authors reported that participants were randomised and if the trial was published as a full report with the results of the main analyses. We categorised articles according to whether allocation concealment was adequate (the person who executed the allocation sequence was different from the person who recruited participants), inadequate (the person who recruited participants also executed the allocation sequence), or unclear (the article failed to describe how the researchers concealed the allocation). We considered the widely used “sealed envelope” method to be inadequate unless performed by an independent third party. We used a kernel density plot to compare the P values of trials that used adequate concealment methods with those that used inadequate methods; we used P values because these were readily available across most of the trials, which used different statistical methods and outcome measures. Our statistical analyses adjusted for clustering effects by journal.

Among the 234 trials that met the inclusion criteria, allocation concealment was adequate in 132 (56%) and inadequate in 41 (18%); in 61 (26%) the concealment method was unclear. Of the trials whose allocation concealment was considered adequate, 118 used independent allocation (which included using a telephone, fax machine, or pager to a randomisation service); five used sealed envelopes opened by a third party; eight used a computer; and one used a combination of adequate methods. Of the 41 trials whose allocation concealment was inadequate, 39 used sealed envelopes, one selected a card from a pile, and one added the name of the next participant to the randomisation list.

What is already known on this topic

The effect of adequacy of allocation concealment in randomised controlled trials may influence the degree of effect

What this study adds

Despite researchers' acceptance that adequate allocation concealment is important, almost a fifth of trials recently published in major medical journals used inadequate concealment and a quarter failed to describe how the allocation was concealed

For 28 (17%) of the 166 trials published in the CONSORT journals (the BMJ, the Lancet, and JAMA) and 33 (49%) of the 68 trials published in the non-CONSORT journal (the New England Journal of Medicine) we could not discern whether allocation was adequately concealed. We excluded 97 of the original 234 trials from the P value analysis because a single P value was not extractable (for example, unclear primary outcome). The figure shows a trend towards lower P values for inadequately concealed trials (mean = 0.022); P values from trials that used adequate methods were more widely spread around a higher mean value (0.052). The difference between these two mean values was significant (P = 0.045). In a logistic regression, adjusted for sample size and journal, we found that the odds of a trial with inadequate concealment yielding a significant result (P ≤ 0.05) compared with a trial with adequate concealment was 1.8 (95% confidence interval 0.8 to 3.7), thus suggesting that trials using inadequate concealment tend to show significant differences between the groups in the primary outcome more often than trials using adequate concealment.

Figure1

Distribution of P values by adequacy of allocation concealment. As the P values were highly positively skewed, the data were transformed using the logit function. The vertical lines represent mean P values for trials using adequate or inadequate concealment

Comments

We found that despite the CONSORT statement more than 40% of trials published in major medical journals either did not use adequate methods or failed to describe how they concealed the allocation. Our results confirm the association found in previous studies that trials using inadequate allocation concealment are more likely to report significant findings than those using adequate concealment. Readers should critically assess the reported methods of allocation concealment.

Acknowledgments

We thank Doug Altman for reviewing an earlier version of the paper and providing valuable feedback. A list of included studies is available on request from CH.

Footnotes

  • Contributors DJT suggested the idea of the review. CH, DJT, and JW did the searches and extracted data from the included papers. CH, with advice from SH and JMB, undertook the data analysis. CH wrote the paper, and JMB, SH, DJT, and JW commented on it. CH is the guarantor.

  • Funding Department of Health Sciences, University of York. Competing interest statement: None declared

  • Conflict of interest Not needed. doi 10.1136/bmj.38413.576713.AE

  • Ethical approval Not needed. doi 10.1136/bmj.38413.576713.AE

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
View Abstract