In recent years, the inconsistency in Bayesian deep learning has garnered increasing attention. Tempered or generalized posterior distributions often offer a direct and effective solution to this issue. However, understanding the underlying causes and evaluating the effectiveness of generalized posteriors remain active areas of research. In this study, we introduce a unified theoretical framework to attribute Bayesian inconsistency to model misspecification and inadequate priors. We interpret the generalization of the posterior with a temperature factor as a correction for misspecified models through adjustments to the joint probability model, and the recalibration of priors by redistributing probability mass on models within the hypothesis space using data samples. Additionally, we highlight a distinctive feature of Laplace approximation, which ensures that the generalized normalizing constant can be treated as invariant, unlike the typical scenario in general Bayesian learning where this constant varies with model parameters post-generalization. Building on this insight, we propose the generalized Laplace approximation, which involves a simple adjustment to the computation of the Hessian matrix of the regularized loss function. This method offers a flexible and scalable framework for obtaining high-quality posterior distributions. We assess the performance and properties of the generalized Laplace approximation on state-of-the-art neural networks and real-world datasets.

翻译：近年来，贝叶斯深度学习中的不一致性问题日益受到关注。回火或广义后验分布通常为此问题提供直接有效的解决方案。然而，理解其根本原因并评估广义后验的有效性仍是活跃的研究领域。本研究引入一个统一的理论框架，将贝叶斯不一致性归因于模型设定错误和先验分布不足。我们将带温度因子的后验泛化解释为：通过调整联合概率模型来修正设定错误的模型，并利用数据样本在假设空间内重新分配模型上的概率质量以重新校准先验。此外，我们指出拉普拉斯近似的一个独特性质，即它能确保广义归一化常数可被视为不变——这与一般贝叶斯学习中该常数在泛化后随模型参数变化的典型情况不同。基于这一认识，我们提出了广义拉普拉斯近似方法，该方法仅需对正则化损失函数的Hessian矩阵计算进行简单调整。此方法为获取高质量后验分布提供了一个灵活且可扩展的框架。我们在前沿神经网络和真实数据集上评估了广义拉普拉斯近似的性能与特性。