Deep Ensembles, as a type of Bayesian Neural Networks, can be used to estimate uncertainty on the prediction of multiple neural networks by collecting votes from each network and computing the difference in those predictions. In this paper, we introduce a method for uncertainty estimation that considers a set of independent categorical distributions for each layer of the network, giving many more possible samples with overlapped layers than in the regular Deep Ensembles. We further introduce an optimized inference procedure that reuses common layer outputs, achieving up to 19x speed up and reducing memory usage quadratically. We also show that the method can be further improved by ranking samples, resulting in models that require less memory and time to run while achieving higher uncertainty quality than Deep Ensembles.

翻译：深度集成作为一种贝叶斯神经网络，通过收集多个神经网络的投票并计算预测差异来估计预测不确定性。本文提出了一种不确定性估计方法，该方法为网络每一层考虑一组独立的分类分布，与常规深度集成相比，能够生成更多具有重叠层的可能样本。我们进一步提出了一种优化推理过程，通过重用公共层输出实现高达19倍的加速，并将内存使用量降低至二次方级别。此外，我们证明该方法可通过样本排序进一步改进，从而在保持比深度集成更高不确定性质量的同时，降低模型运行所需的内存与时间开销。