Deep Ensembles, as a type of Bayesian Neural Networks, can be used to estimate uncertainty on the prediction of multiple neural networks by collecting votes from each network and computing the difference in those predictions. In this paper, we introduce a method for uncertainty estimation that considers a set of independent categorical distributions for each layer of the network, giving many more possible samples with overlapped layers than in the regular Deep Ensembles. We further introduce an optimized inference procedure that reuses common layer outputs, achieving up to 19x speed up and reducing memory usage quadratically. We also show that the method can be further improved by ranking samples, resulting in models that require less memory and time to run while achieving higher uncertainty quality than Deep Ensembles.

翻译：深度集成（Deep Ensembles）作为贝叶斯神经网络的一种形式，通过收集多个神经网络的投票并计算预测差异，可用于估计多个神经网络预测的不确定性。本文提出一种不确定性估计方法，该方法考虑网络每一层各独立分类分布的集合，相较于常规深度集成，能够生成更多具有重叠层的可能样本。我们进一步引入一种优化推理过程，通过复用公共层输出，实现最高19倍的速度提升并呈二次方地降低内存使用。研究还表明，通过对样本进行排序可进一步改进该方法，所得模型在降低内存占用与运行时间的同时，能获得比深度集成更优的不确定性质量。