Bayesian model comparison (BMC) offers a principled probabilistic approach to study and rank competing models. In standard BMC, we construct a discrete probability distribution over the set of possible models, conditional on the observed data of interest. These posterior model probabilities (PMPs) are measures of uncertainty, but -- when derived from a finite number of observations -- are also uncertain themselves. In this paper, we conceptualize distinct levels of uncertainty which arise in BMC. We explore a fully probabilistic framework for quantifying meta-uncertainty, resulting in an applied method to enhance any BMC workflow. Drawing on both Bayesian and frequentist techniques, we represent the uncertainty over the uncertain PMPs via meta-models which combine simulated and observed data into a predictive distribution for PMPs on new data. We demonstrate the utility of the proposed method in the context of conjugate Bayesian regression, likelihood-based inference with Markov chain Monte Carlo, and simulation-based inference with neural networks.

翻译：贝叶斯模型比较（BMC）提供了一种基于原则的概率方法来研究与排序竞争模型。在标准贝叶斯模型比较中，我们根据观测数据构建一个关于可能模型集合的离散概率分布。这些后验模型概率（PMPs）本身是不确定性的度量，但由于基于有限观测数据推导得出，它们自身也带有不确定性。本文对贝叶斯模型比较中出现的不同层次的不确定性进行了概念化，并探索了一种量化元不确定性的全概率框架，最终形成了一种可增强任何贝叶斯模型比较工作流程的实用方法。通过融合贝叶斯与频率学派技术，我们利用元模型将模拟数据与观测数据相结合，形成针对新数据上的后验模型概率的预测分布，从而表征后验模型概率的不确定性。我们分别在共轭贝叶斯回归、基于马尔可夫链蒙特卡洛的似然推断以及基于神经网络的模拟推断三个场景中，展示了所提出方法的有效性。