Bayesian inference provides a methodology for parameter estimation and uncertainty quantification in machine learning and deep learning methods. Variational inference and Markov Chain Monte-Carlo (MCMC) sampling techniques are used to implement Bayesian inference. In the past three decades, MCMC methods have faced a number of challenges in being adapted to larger models (such as in deep learning) and big data problems. Advanced proposals that incorporate gradients, such as a Langevin proposal distribution, provide a means to address some of the limitations of MCMC sampling for Bayesian neural networks. Furthermore, MCMC methods have typically been constrained to use by statisticians and are still not prominent among deep learning researchers. We present a tutorial for MCMC methods that covers simple Bayesian linear and logistic models, and Bayesian neural networks. The aim of this tutorial is to bridge the gap between theory and implementation via coding, given a general sparsity of libraries and tutorials to this end. This tutorial provides code in Python with data and instructions that enable their use and extension. We provide results for some benchmark problems showing the strengths and weaknesses of implementing the respective Bayesian models via MCMC. We highlight the challenges in sampling multi-modal posterior distributions in particular for the case of Bayesian neural networks, and the need for further improvement of convergence diagnosis.

翻译：贝叶斯推理为机器学习和深度学习方法的参数估计与不确定性量化提供了一种方法论。变分推理和马尔可夫链蒙特卡洛（MCMC）采样技术被用于实现贝叶斯推理。在过去三十年中，MCMC方法在适应更大规模模型（如深度学习）和大数据问题时面临诸多挑战。包含梯度的高级提议机制（如朗之万提议分布）为解决贝叶斯神经网络中MCMC采样的部分局限性提供了手段。此外，MCMC方法通常局限于统计学家使用，在深度学习研究者中仍不普遍。我们提供了一份MCMC方法教程，涵盖简单的贝叶斯线性模型、逻辑模型以及贝叶斯神经网络。本教程旨在通过编码实践弥合理论与实现之间的鸿沟，鉴于目前相关库和教程普遍缺乏的现状。本教程提供了Python代码，附带数据和操作说明，便于用户使用和扩展。我们给出了若干基准问题的结果，展示了通过MCMC实现相应贝叶斯模型的优势与不足。我们特别强调了在贝叶斯神经网络情况下对多模态后验分布进行采样的挑战，以及进一步改进收敛性诊断的必要性。