Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.

翻译：经典变分推断旨在寻找贝叶斯后验分布的最优近似，然而在模型设定错误的情况下，即使是精确贝叶斯后验也往往缺乏实际意义。我们提出预测变分推断（PVI）：一个通用推断框架，通过从预测最优的后验密度中进行采样，使得由此得到的后验预测分布尽可能逼近真实数据生成过程，并以多种评分规则衡量这种逼近程度。通过优化目标函数，预测变分推断通常不等同于（甚至不试图近似）贝叶斯后验，即使在大样本极限下也是如此。相反，我们将其解释为隐式分层扩展。此外，学习到的后验不确定性能够检测参数在总体中的异质性，从而实现自动化模型诊断。该框架同时适用于精确似然模型和似然不可得模型。我们通过实际数据示例展示了其应用。