We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distributions and trained to approximate the Bayesian posterior with variational inference. We implement stochastic ensembles based on Monte Carlo dropout, DropConnect and a novel non-parametric version of dropout and evaluate them on a toy problem and CIFAR image classification. For both tasks, we test the quality of the posteriors directly against Hamiltonian Monte Carlo simulations. Our results show that stochastic ensembles provide more accurate posterior estimates than other popular baselines for Bayesian inference.

翻译：我们引入随机神经网络集成来近似贝叶斯后验，将dropout等随机方法与深度集成相结合。随机集成被表述为分布族，并通过变分推断训练以逼近贝叶斯后验。我们基于蒙特卡洛dropout、DropConnect以及一种新型的非参数化dropout版本实现了随机集成，并在玩具问题与CIFAR图像分类任务上进行了评估。针对这两项任务，我们直接通过哈密顿蒙特卡洛模拟测试了后验质量。结果表明，与其他流行的贝叶斯推断基线方法相比，随机集成能够提供更准确的后验估计。