TY - JOUR
T1 - What difference might retractions make? An estimate of the potential epistemic cost of retractions on meta-analyses
AU - Fanelli, Daniele
AU - Wong, Julie
AU - Moher, David
N1 - Funding Information:
A clearer case may be made that retractions entail a waste of resources. However, the financial costs are surprisingly contained. A recent study, in particular, tried to estimate the financial impact of studies that were funded by the US National Institutes of Health and were retracted due to findings of misconduct. It concluded that the costs amount, in the least conservative estimates, to between 0.01% and 0.05% of the NIH budget, a figure that was deemed low in the authors’ own assessment (Stern et al. ). A prior case-study analysis conducted in 2010 concluded that, if the costs that were estimated for that case were extrapolated to all of the allegations made to the US Office of Research Integrity in their last reporting year, the costs of conducting all of those investigations would amount to 110 million US dollars (Michalek et al. ). This is clearly an implausible worse-case scenario, since most allegations do not require the same amount of investigation, if any at all, and yet the costs it suggests are only circa 0.35% of the NIH budget for 2010 (NIH ).
Publisher Copyright:
© 2021 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
PY - 2022/10/3
Y1 - 2022/10/3
N2 - The extent to which a retraction might require revising previous scientific estimates and beliefs–which we define as the epistemic cost–is unknown. We collected a sample of 229 meta-analyses published between 2013 and 2016 that had cited a retracted study, assessed whether this study was included in the meta-analytic estimate and, if so, re-calculated the summary effect size without it. The majority (68% of N = 229) of retractions had occurred at least one year prior to the publication of the citing meta-analysis. In 53% of these avoidable citations, the retracted study was cited as a candidate for inclusion, and only in 34% of these meta-analyses (13% of total) the study was explicitly excluded because it had been retracted. Meta-analyses that included retracted studies were published in journals with significantly lower impact factor. Summary estimates without the retracted study were lower than the original if the retraction was due to issues with data or results and higher otherwise, but the effect was small. We conclude that meta-analyses have a problematically high probability of citing retracted articles and of including them in their pooled summaries, but the overall epistemic cost is contained.
AB - The extent to which a retraction might require revising previous scientific estimates and beliefs–which we define as the epistemic cost–is unknown. We collected a sample of 229 meta-analyses published between 2013 and 2016 that had cited a retracted study, assessed whether this study was included in the meta-analytic estimate and, if so, re-calculated the summary effect size without it. The majority (68% of N = 229) of retractions had occurred at least one year prior to the publication of the citing meta-analysis. In 53% of these avoidable citations, the retracted study was cited as a candidate for inclusion, and only in 34% of these meta-analyses (13% of total) the study was explicitly excluded because it had been retracted. Meta-analyses that included retracted studies were published in journals with significantly lower impact factor. Summary estimates without the retracted study were lower than the original if the retraction was due to issues with data or results and higher otherwise, but the effect was small. We conclude that meta-analyses have a problematically high probability of citing retracted articles and of including them in their pooled summaries, but the overall epistemic cost is contained.
UR - http://www.scopus.com/inward/record.url?scp=85110886891&partnerID=8YFLogxK
U2 - 10.1080/08989621.2021.1947810
DO - 10.1080/08989621.2021.1947810
M3 - Article
C2 - 34196235
AN - SCOPUS:85110886891
SN - 0898-9621
VL - 29
SP - 442
EP - 459
JO - Accountability in Research
JF - Accountability in Research
IS - 7
ER -