The last decade witnessed a growing interest in Bayesian learning. Yet, the technicality of the topic and the multitude of ingredients involved therein, besides the complexity of turning theory into practical implementations, limit the use of the Bayesian learning paradigm, preventing its widespread adoption across different fields and applications. This self-contained survey engages and introduces readers to the principles and algorithms of Bayesian Learning for Neural Networks. It provides an introduction to the topic from an accessible, practical-algorithmic perspective. Upon providing a general introduction to Bayesian Neural Networks, we discuss and present both standard and recent approaches for Bayesian inference, with an emphasis on solutions relying on Variational Inference and the use of Natural gradients. We also discuss the use of manifold optimization as a state-of-the-art approach to Bayesian learning. We examine the characteristic properties of all the discussed methods, and provide pseudo-codes for their implementation, paying attention to practical aspects, such as the computation of the gradients.

翻译：过去十年中，贝叶斯学习日益受到关注。然而，该主题的专业性、其中涉及的众多要素，以及将理论转化为实际实现的复杂性，限制了贝叶斯学习范式的应用，阻碍其在不同领域和场景中的广泛采用。本篇自包含的综述向读者介绍并引导理解神经网络贝叶斯学习的原理与算法。我们从简明易懂的实用算法视角对该主题进行概述。在给出贝叶斯神经网络的一般性介绍后，我们讨论并呈现了贝叶斯推理的标准方法与最新方法，重点强调依赖于变分推理和自然梯度的解决方案。我们还讨论了将流形优化作为贝叶斯学习前沿方法的应用。我们审视了所有讨论方法的特性，并提供了其实现的伪代码，同时关注实际方面，例如梯度的计算。