Machine learning is increasingly deployed in safety-critical domains where robustness against adversarial attacks is crucial and erroneous predictions could lead to potentially catastrophic consequences. This highlights the need for learning systems to be equipped with the means to determine a model's confidence in its prediction and the epistemic uncertainty associated with it, 'to know when a model does not know'. In this paper, we propose a novel Random-Set Convolutional Neural Network (RS-CNN) for classification which predicts belief functions rather than probability vectors over the set of classes, using the mathematics of random sets, i.e., distributions over the power set of the sample space. Based on the epistemic deep learning approach, random-set models are capable of representing the 'epistemic' uncertainty induced in machine learning by limited training sets. We estimate epistemic uncertainty by approximating the size of credal sets associated with the predicted belief functions, and experimentally demonstrate how our approach outperforms competing uncertainty-aware approaches in a classical evaluation setting. The performance of RS-CNN is best demonstrated on OOD samples where it manages to capture the true prediction while standard CNNs fail.

翻译：机器学习越来越多地部署在安全关键领域，在这些领域中，抵御对抗性攻击的鲁棒性至关重要，而错误的预测可能导致潜在的灾难性后果。这凸显了学习系统需要配备能够确定模型对其预测的置信度及其相关认知不确定性的手段，即“知道模型何时不知道”。本文提出了一种新颖的随机集卷积神经网络（RS-CNN）用于分类，它利用随机集数学（即样本空间幂集上的分布）预测信念函数，而非类别集上的概率向量。基于认知深度学习方法，随机集模型能够表示由有限训练集在机器学习中引发的“认知”不确定性。我们通过近似与预测信念函数相关的信度集大小来估计认知不确定性，并通过实验证明我们的方法在经典评估设置中性能优于其他不确定性感知方法。RS-CNN的性能在分布外样本上得到最佳体现，在此类样本上它能成功捕捉真实预测，而标准CNN则无法做到。