Meta-analyses triggered by previous (false-)significant findings: Problems and solutions

Ewoud Schuit, Kit C B Roes, Ben W J Mol, Anneke Kwee, Karel G M Moons, Rolf H H Groenwold

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Background: Meta-analyses are typically triggered by a (potentially false-significant) finding in one of the preceding primary studies. We studied consequences of meta-analysis investigating effects when primary studies that triggered such meta-analysis are also included. Methods: We analytically determined the bias of the treatment effect estimates obtained by meta-analysis, conditional on the number of included primary and false-significant studies. The type I error rate and power of the meta-analysis were assessed using simulations. We applied a method for bias-correction, by subtracting an analytically derived bias from the treatment effect estimated in meta-analysis. Results: Bias in meta-analytical effects and type I error rates increased when increasing numbers of primary studies with false-significant effects were included. When 20% of the primary studies showed false-significant effects, the bias was 0.33 (z-score) instead of 0, and the type I error rate was 23% instead of 5%. After applying a bias-correction, the type I error rate became indeed 5%. Conclusions: Inclusion of primary studies with false-significant effects leads to biased effect estimates and inflated type I error rates in the meta-analysis, depending on the number of false-significant studies. This bias can be adjusted for.

LanguageEnglish
Article number57
JournalSystematic Reviews
Volume4
Issue number1
DOIs
Publication statusPublished - 25 Apr 2015
Externally publishedYes

Keywords

  • Bias
  • Meta-analysis
  • Power
  • Type I error

ASJC Scopus subject areas

  • Medicine (miscellaneous)

Cite this

Schuit, E., Roes, K. C. B., Mol, B. W. J., Kwee, A., Moons, K. G. M., & Groenwold, R. H. H. (2015). Meta-analyses triggered by previous (false-)significant findings: Problems and solutions. Systematic Reviews, 4(1), [57]. https://doi.org/10.1186/s13643-015-0048-9