Bayesian Neural Networks (BNNs) offer a probabilistic interpretation for deep learning models by imposing a prior distribution over model parameters and inferencing a posterior distribution based on observed data. The model sampled from the posterior distribution can be used for providing ensemble predictions and quantifying prediction uncertainty. It is well-known that deep learning models with a lower sharpness have a better generalization ability. Nonetheless, existing posterior inferences are not aware of sharpness/flatness, hence possibly leading to high sharpness for the models sampled from it. In this paper, we develop theories, the Bayesian setting, and the variational inference approach for the sharpness-aware posterior. Specifically, the models sampled from our sharpness-aware posterior and the optimal approximate posterior estimating this sharpness-aware posterior have a better flatness, hence possibly possessing a higher generalization ability. We conduct experiments by leveraging the sharpness-aware posterior with the state-of-the-art Bayesian Neural Networks, showing that the flat-seeking counterparts outperform their baselines in all metrics of interest.

翻译：贝叶斯神经网络（BNNs）通过对模型参数施加先验分布，并基于观测数据推断后验分布，为深度学习模型提供了一种概率解释。从后验分布中采样的模型可用于提供集成预测并量化预测不确定性。众所周知，具有更低锐度的深度学习模型具有更好的泛化能力。然而，现有的后验推断并未考虑锐度/平坦度，因此可能导致从中采样的模型具有高锐度。本文针对锐度感知后验发展了理论、贝叶斯设定及变分推断方法。具体而言，从我们的锐度感知后验中采样的模型以及估计该锐度感知后验的最优近似后验具有更好的平坦度，因此可能具备更高的泛化能力。我们将锐度感知后验与最先进的贝叶斯神经网络相结合进行实验，结果表明，在各项感兴趣指标上，寻求平坦的对应方法均优于其基线方法。