Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

František Bartoš*, Maximilian Maier, Eric Jan Wagenmakers, Franziska Nippold, Hristos Doucouliagos, John P. A. Ioannidis, Willem M. Otte, Martina Sladekova, Teshome K. Deresssa, Stephan B. Bruns, Daniele Fanelli, T. D. Stanley

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

29 Downloads (Pure)

Abstract

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Original languageEnglish
Pages (from-to)500-511
Number of pages12
JournalResearch Synthesis Methods
Volume15
Issue number3
Early online date7 Feb 2024
DOIs
Publication statusPublished - May 2024

Keywords

  • Bayesian
  • effect sizes
  • evidence
  • meta-analysis
  • model-averaging
  • publication bias
  • RoBMA

ASJC Scopus subject areas

  • Education

Fingerprint

Dive into the research topics of 'Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics'. Together they form a unique fingerprint.

Cite this