Bayesian neural networks (BNNs) have received an increased interest in the last years. In BNNs, a complete posterior distribution of the unknown weight and bias parameters of the network is produced during the training stage. This probabilistic estimation offers several advantages with respect to point-wise estimates, in particular, the ability to provide uncertainty quantification when predicting new data. This feature inherent to the Bayesian paradigm, is useful in countless machine learning applications. It is particularly appealing in areas where decision-making has a crucial impact, such as medical healthcare or autonomous driving. The main challenge of BNNs is the computational cost of the training procedure since Bayesian techniques often face a severe curse of dimensionality. Adaptive importance sampling (AIS) is one of the most prominent Monte Carlo methodologies benefiting from sounded convergence guarantees and ease for adaptation. This work aims to show that AIS constitutes a successful approach for designing BNNs. More precisely, we propose a novel algorithm PMCnet that includes an efficient adaptation mechanism, exploiting geometric information on the complex (often multimodal) posterior distribution. Numerical results illustrate the excellent performance and the improved exploration capabilities of the proposed method for both shallow and deep neural networks.

翻译：贝叶斯神经网络（BNNs）在过去几年中受到越来越多的关注。在BNNs中，训练阶段会产生网络未知权重和偏置参数的完整后验分布。与点估计相比，这种概率估计具有若干优势，尤其是能够在新数据预测时提供不确定性量化。这一源于贝叶斯范式的特性在众多机器学习应用中极具价值，在医疗健康或自动驾驶等决策具有关键影响的领域尤为突出。BNNs的主要挑战在于训练过程的计算成本，因为贝叶斯方法往往面临严重的维度灾难。自适应重要性采样（AIS）是最著名的蒙特卡洛方法之一，具有可靠的收敛保证和易于调整的优势。本研究旨在证明AIS是设计BNNs的成功途径。具体而言，我们提出了一种新颖的算法PMCnet，该算法包含高效的适应机制，能利用复杂（通常为多模态）后验分布的几何信息。数值结果表明，所提方法在浅层和深度神经网络中均展现出卓越的性能和更强的探索能力。