Bayesian Neural Networks (BNNs) offer a probabilistic interpretation for deep learning models by imposing a prior distribution over model parameters and inferencing a posterior distribution based on observed data. The model sampled from the posterior distribution can be used for providing ensemble predictions and quantifying prediction uncertainty. It is well-known that deep learning models with a lower sharpness have a better generalization ability. Nonetheless, existing posterior inferences are not aware of sharpness/flatness, hence possibly leading to high sharpness for the models sampled from it. In this paper, we develop theories, the Bayesian setting, and the variational inference approach for the sharpness-aware posterior. Specifically, the models sampled from our sharpness-aware posterior and the optimal approximate posterior estimating this sharpness-aware posterior have a better flatness, hence possibly possessing a higher generalization ability. We conduct experiments by leveraging the sharpness-aware posterior with the state-of-the-art Bayesian Neural Networks, showing that the flat-seeking counterparts outperform their baselines in all metrics of interest.

翻译：贝叶斯神经网络通过在模型参数上施加先验分布，并基于观测数据推断后验分布，为深度学习模型提供了概率解释。从后验分布中采样得到的模型可用于集成预测和量化预测不确定性。众所周知，具有更低锐度的深度学习模型具有更好的泛化能力。然而，现有后验推断方法未考虑锐度/平坦性，因此可能导致从中采样的模型具有高锐度。本文针对锐度感知后验发展了理论框架、贝叶斯设置和变分推断方法。具体而言，从我们的锐度感知后验采样得到的模型，以及估计该锐度感知后验的最优近似后验，具有更好的平坦性，因此可能拥有更高的泛化能力。我们通过将锐度感知后验与最先进的贝叶斯神经网络相结合进行实验，结果表明，所有评估指标中，平坦性导向的对应模型均优于基准方法。