Posterior predictive p-values (ppps) have become popular tools for Bayesian model assessment, being general-purpose and easy to use. However, interpretation can be difficult because their distribution is not uniform under the hypothesis that the model did generate the data. Calibrated ppps (cppps) can be obtained via a bootstrap-like procedure, yet remain unavailable in practice due to high computational cost. This paper introduces methods to enable efficient approximation of cppps and their uncertainty for fast model assessment. We first investigate the computational trade-off between the number of calibration replicates and the number of MCMC samples per replicate. Provided that the MCMC chain from the real data has converged, using short MCMC chains per calibration replicate can save significant computation time compared to naive implementations, without significant loss in accuracy. We propose different variance estimators for the cppp approximation, which can be used to confirm quickly the lack of evidence against model misspecification. As variance estimation uses effective sample sizes of many short MCMC chains, we show these can be approximated well from the real-data MCMC chain. The procedure for cppp is implemented in NIMBLE, a flexible framework for hierarchical modeling that supports many models and discrepancy measures.

翻译：后验预测p值（ppps）已成为贝叶斯模型评估的常用工具，因其通用性强且易于使用而广受欢迎。然而，当模型确实生成数据时，其分布并非均匀分布，导致解释存在困难。校准后的ppp（cppp）可通过类自助法流程获得，但因计算成本过高而难以实际应用。本文提出一系列方法，旨在高效近似cppp及其不确定性，以实现快速模型评估。我们首先研究了校准重复次数与每次重复中MCMC样本数之间的计算权衡。只要基于真实数据的MCMC链已收敛，与原始实现相比，在校准重复中使用短MCMC链可在不显著损失精度的情况下大幅节省计算时间。我们提出了cppp近似方差的不同估计量，这些估计量可用于快速确认模型设定错误缺乏证据。由于方差估计需使用大量短MCMC链的有效样本量，我们证明这些有效样本量可通过真实数据MCMC链进行良好近似。cppp流程已在NIMBLE（一种支持多种模型与差异度量的灵活分层建模框架）中实现。