The large-scale pre-trained neural network has achieved notable success in enhancing performance for downstream tasks. Another promising approach for generalization is Bayesian Neural Network (BNN), which integrates Bayesian methods into neural network architectures, offering advantages such as Bayesian Model averaging (BMA) and uncertainty quantification. Despite these benefits, transfer learning for BNNs has not been widely investigated and shows limited improvement. We hypothesize that this issue arises from the inability to find flat minima, which is crucial for generalization performance. To address this, we evaluate the sharpness of BNNs in various settings, revealing their insufficiency in seeking flat minima and the influence of flatness on BMA performance. Therefore, we propose Sharpness-aware Bayesian Model Averaging (SA-BMA), a Bayesian-fitting flat posterior seeking optimizer integrated with Bayesian transfer learning. SA-BMA calculates the divergence between posteriors in the parameter space, aligning with the nature of BNNs, and serves as a generalized version of existing sharpness-aware optimizers. We validate that SA-BMA improves generalization performance in few-shot classification and distribution shift scenarios by ensuring flatness.

翻译：大规模预训练神经网络在提升下游任务性能方面已取得显著成功。另一种具有前景的泛化方法是贝叶斯神经网络（BNN），它将贝叶斯方法融入神经网络架构，提供了贝叶斯模型平均（BMA）和不确定性量化等优势。尽管存在这些优点，BNN的迁移学习尚未得到广泛研究且改进有限。我们假设该问题源于难以找到对泛化性能至关重要的平坦极小值。为此，我们在多种设置下评估了BNN的锐度，揭示了其在寻求平坦极小值方面的不足以及平坦性对BMA性能的影响。因此，我们提出了锐度感知贝叶斯模型平均（SA-BMA），这是一种与贝叶斯迁移学习相结合的、寻求贝叶斯拟合平坦后验的优化器。SA-BMA计算参数空间中后验分布的散度，这与BNN的特性相符，并可作为现有锐度感知优化器的广义版本。我们验证了SA-BMA通过确保平坦性，在少样本分类和分布偏移场景中提升了泛化性能。