In meta-analyses of proportion studies, funnel plots were found to be an inaccurate method of assessing publication bias

Objective: To assess the utility of funnel plots in assessing publication bias (PB) in meta-analyses of proportion studies.

Study design and setting: Meta-analysis simulation study and meta-analysis of published literature reporting peri-operative mortality after abdominal aortic aneurysm (AAA) repair. Data for the simulation study were stochastically generated. A literature search of Medline and Embase was performed to identify studies for inclusion in the published literature meta-analyses.

Results: The simulation study demonstrated that conventionally constructed funnel plots (log odds vs. 1/standard error [1/SE]) for extreme proportional outcomes were asymmetric despite no PB. Alternative funnel plots constructed using study size rather than 1/SE showed no asymmetry for extreme proportional outcomes. When used in meta-analyses of the mortality of AAA repair, these alternative funnel plots highlighted the possibility for conventional funnel plots to demonstrate asymmetry when there was no evidence of PB.

Conclusion: Conventional funnel plots used to assess for potential PB in meta-analyses are inaccurate for meta-analyses of proportion studies with low proportion outcomes. Funnel plots of study size against log odds may be a more accurate way of assessing for PB in these studies.