Bayesian inference in deep neural networks is challenging due to the high-dimensional, strongly multi-modal parameter posterior density landscape. Markov chain Monte Carlo approaches asymptotically recover the true posterior but are considered prohibitively expensive for large modern architectures. Local methods, which have emerged as a popular alternative, focus on specific parameter regions that can be approximated by functions with tractable integrals. While these often yield satisfactory empirical results, they fail, by definition, to account for the multi-modality of the parameter posterior. In this work, we argue that the dilemma between exact-but-unaffordable and cheap-but-inexact approaches can be mitigated by exploiting symmetries in the posterior landscape. Such symmetries, induced by neuron interchangeability and certain activation functions, manifest in different parameter values leading to the same functional output value. We show theoretically that the posterior predictive density in Bayesian neural networks can be restricted to a symmetry-free parameter reference set. By further deriving an upper bound on the number of Monte Carlo chains required to capture the functional diversity, we propose a straightforward approach for feasible Bayesian inference. Our experiments suggest that efficient sampling is indeed possible, opening up a promising path to accurate uncertainty quantification in deep learning.

翻译：深度神经网络中的贝叶斯推断因参数后验密度呈现高维且强多模态特性而极具挑战性。马尔可夫链蒙特卡洛方法虽能渐近恢复真实后验分布，但针对大规模现代架构被认为计算成本过高。作为主流替代方案的局部方法聚焦于可通过具有可解析积分形式的函数近似的特定参数区域。尽管此类方法常产生满意的经验结果，但其本质上无法刻画参数后验的多模态性。本文论证，通过利用后验景观中的对称性，可缓解精确但不可负担与廉价但不精确方法之间的两难困境。由神经元可互换性及特定激活函数引发的此类对称性，表现为不同参数值产生相同函数输出值。我们从理论上证明，贝叶斯神经网络的后验预测密度可被限制在无对称性参数参考集内。通过进一步推导捕获函数多样性所需蒙特卡洛链数量的上界，我们提出一种实现可行贝叶斯推断的简洁方法。实验表明，高效采样确实可行，为深度学习中的精确不确定性量化开辟了有前景的途径。