We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distributions and trained to approximate the Bayesian posterior with variational inference. We implement stochastic ensembles based on Monte Carlo dropout, DropConnect and a novel non-parametric version of dropout and evaluate them on a toy problem and CIFAR image classification. For both tasks, we test the quality of the posteriors directly against Hamiltonian Monte Carlo simulations. Our results show that stochastic ensembles provide more accurate posterior estimates than other popular baselines for Bayesian inference.

翻译：我们引入随机神经网络集成来近似贝叶斯后验，将dropout等随机方法与深度集成相结合。随机集成被定义为分布族，并通过变分推断进行训练以逼近贝叶斯后验。我们基于蒙特卡洛dropout、DropConnect以及一种新型非参数化dropout实现了随机集成，并在一个玩具问题与CIFAR图像分类任务上进行了评估。针对这两个任务，我们直接与哈密顿蒙特卡洛模拟对比，检验后验质量。结果表明，相较于其他主流的贝叶斯推断基线方法，随机集成能提供更准确的后验估计。