Under model misspecification, it is known that Bayesian posteriors often do not properly quantify uncertainty about true or pseudo-true parameters. Even more fundamentally, misspecification leads to a lack of reproducibility in the sense that the same model will yield contradictory posteriors on independent data sets from the true distribution. To define a criterion for reproducible uncertainty quantification under misspecification, we consider the probability that two confidence sets constructed from independent data sets have nonempty overlap, and we establish a lower bound on this overlap probability that holds for any valid confidence sets. We prove that credible sets from the standard posterior can strongly violate this bound, particularly in high-dimensional settings (i.e., with dimension increasing with sample size), indicating that it is not internally coherent under misspecification. To improve reproducibility in an easy-to-use and widely applicable way, we propose to apply bagging to the Bayesian posterior ("BayesBag"'); that is, to use the average of posterior distributions conditioned on bootstrapped datasets. We motivate BayesBag from first principles based on Jeffrey conditionalization and show that the bagged posterior typically satisfies the overlap lower bound. Further, we prove a Bernstein--Von Mises theorem for the bagged posterior, establishing its asymptotic normal distribution. We demonstrate the benefits of BayesBag via simulation experiments and an application to crime rate prediction.

翻译：在模型误设定条件下，已知贝叶斯后验往往无法正确量化真实参数或伪真实参数的不确定性。更根本的是，误设定会导致可复现性缺失，即同一模型在来自真实分布的独立数据集上会产生相互矛盾的后验分布。为定义误设定下可复现不确定性量化的准则，我们考虑基于独立数据集构建的两个置信集具有非空交集的概率，并建立该交集概率的下界——该下界对任意有效置信集均成立。我们证明标准后验下的可信集会严重违反该下界（尤其在维度随样本量增加的高维场景中），表明其在内一致性方面存在缺陷。为以简便易用且广泛适用的方式提升可复现性，我们提出对贝叶斯后验应用袋装法（"BayesBag"），即对自举数据集的后验分布取平均。我们基于杰弗里条件化原理从第一性原理出发推导BayesBag，并证明袋装后验通常满足交集下界。进一步，我们为袋装后验建立了伯恩斯坦-冯·米塞斯定理，证明其渐近正态分布。通过模拟实验和犯罪率预测应用，我们展示了BayesBag的优势。