Scientific claims gain credibility by replicability, especially if replication under different circumstances and varying designs yields equivalent results. Aggregating results over multiple studies is, however, not straightforward, and when the heterogeneity between studies increases, conventional methods such as (Bayesian) meta-analysis and Bayesian sequential updating become infeasible. *Bayesian Evidence Synthesis*, built upon the foundations of the Bayes factor, allows to aggregate support for conceptually similar hypotheses over studies, regardless of methodological differences. We assess the performance of Bayesian Evidence Synthesis over multiple effect and sample sizes, with a broad set of (inequality-constrained) hypotheses using Monte Carlo simulations, focusing explicitly on the complexity of the hypotheses under consideration. The simulations show that this method can evaluate complex (informative) hypotheses regardless of methodological differences between studies, and performs adequately if the set of studies considered has sufficient statistical power. Additionally, we pinpoint challenging conditions that can lead to unsatisfactory results, and provide suggestions on handling these situations. Ultimately, we show that Bayesian Evidence Synthesis is a promising tool that can be used when traditional research synthesis methods are not applicable due to insurmountable between-study heterogeneity.
翻译:科学主张的可信度取决于其可重复性,尤其是在不同条件和不同设计下进行重复实验仍能获得等效结果时。然而,将多项研究的结果汇总并非易事,随着研究间异质性的增加,传统方法(如贝叶斯元分析和贝叶斯序贯更新)变得不可行。基于贝叶斯因子基础构建的*贝叶斯证据综合*方法,允许跨研究聚合概念相似假设的支持度,而无需考虑方法论差异。我们通过蒙特卡罗模拟,在多种效应量和样本量条件下,结合广泛的(不等式约束)假设评估了贝叶斯证据综合的性能,并特别关注所考虑假设的复杂度。模拟结果表明,该方法能评估复杂(信息性)假设,且不受研究间方法论差异的影响,并在考虑的研究集具备足够统计功效时表现良好。此外,我们明确了可能导致不满意结果的挑战性条件,并提供了应对这些情况的建议。最终,我们证明贝叶斯证据综合是一种有前景的工具,可在传统研究综合方法因难以克服的研究间异质性而失效时发挥作用。