We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distributions and trained to approximate the Bayesian posterior with variational inference. We implement stochastic ensembles based on Monte Carlo dropout, DropConnect and a novel non-parametric version of dropout and evaluate them on a toy problem and CIFAR image classification. For CIFAR, the stochastic ensembles are quantitatively compared to published Hamiltonian Monte Carlo results for a ResNet-20 architecture. We also test the quality of the posteriors directly against Hamiltonian Monte Carlo simulations in a simplified toy model. Our results show that in a number of settings, stochastic ensembles provide more accurate posterior estimates than regular deep ensembles.

翻译：我们引入了各种随机神经网络,以接近贝耶斯后方,将辍学和深层集合等随机方法结合起来。这些随机集合是按分布式组合制成的,并经过培训,以变推法来接近巴伊西亚后方;我们根据蒙特卡洛的辍学、DroutConnect和新颖的非参数式的辍学组合以及CIFAR的图像分类来评估它们。对于CIFAR来说,对于CIFAR来说,随机集合与出版的ResNet-20结构的汉密尔顿蒙特卡洛结果相比,是量化的。我们还用一个简化的玩具模型,直接对汉密尔顿蒙特卡洛模拟的后方质量进行测试。我们的结果显示,在许多环境中,随机组合比正常的深层组合提供更准确的后方估计。