Posterior predictive p-values (ppps) have become popular tools for Bayesian model criticism, being general-purpose and easy to use. However, their interpretation can be difficult because their distribution is not uniform under the hypothesis that the model did generate the data. To address this issue, procedures to obtain calibrated ppps (cppps) have been proposed although not used in practice, because they require repeated simulation of new data and model estimation via MCMC. Here we give methods to balance the computational trade-off between the number of calibration replicates and the number of MCMC samples per replicate. Our results suggest that investing in a large number of calibration replicates while using short MCMC chains can save significant computation time compared to naive implementations, without significant loss in accuracy. We propose different estimators for the variance of the cppp that can be used to confirm quickly when the model fits the data well. Variance estimation requires the effective sample sizes of many short MCMC chains; we show that these can be well approximated using the single long MCMC chain from the real-data model. The procedure for cppp is implemented in NIMBLE, a flexible framework for hierarchical modeling that supports many models and discrepancy measures.

翻译：后验预测p值（后验预测p值，简称ppps）作为通用且易于使用的工具，在贝叶斯模型批评中已广受欢迎。然而，其解释存在困难，因为当模型确实生成数据时，这些p值的分布并非均匀分布。为解决此问题，虽已提出获得校准后验预测p值（校准后验预测p值，简称cppps）的程序，但由于需要重复模拟新数据并通过MCMC进行模型估计，这些程序在实践中尚未被采用。本文提出平衡校准重复次数与每次重复中MCMC样本数之间计算权衡的方法。结果表明，与朴素实现相比，采用较短的MCMC链并增加校准重复次数，可在不显著损失精度的情况下大幅节省计算时间。我们提出了cppp方差的不同估计量，可用于快速确认模型与数据的拟合情况。方差估计需要大量短MCMC链的有效样本量；我们证明，这些有效样本量可通过真实数据模型中的单条长MCMC链进行良好近似。cppp计算程序已在NIMBLE中实现，这是一个支持多种模型与差异度量的灵活层次建模框架。