Out-of-distribution (OOD) detection is critical for preventing deep learning models from making incorrect predictions to ensure the safety of artificial intelligence systems. Especially in safety-critical applications such as medical diagnosis and autonomous driving, the cost of incorrect decisions is usually unbearable. However, neural networks often suffer from the overconfidence issue, making high confidence for OOD data which are never seen during training process and may be irrelevant to training data, namely in-distribution (ID) data. Determining the reliability of the prediction is still a difficult and challenging task. In this work, we propose Uncertainty-Estimation with Normalized Logits (UE-NL), a robust learning method for OOD detection, which has three main benefits. (1) Neural networks with UE-NL treat every ID sample equally by predicting the uncertainty score of input data and the uncertainty is added into softmax function to adjust the learning strength of easy and hard samples during training phase, making the model learn robustly and accurately. (2) UE-NL enforces a constant vector norm on the logits to decouple the effect of the increasing output norm from optimization process, which causes the overconfidence issue to some extent. (3) UE-NL provides a new metric, the magnitude of uncertainty score, to detect OOD data. Experiments demonstrate that UE-NL achieves top performance on common OOD benchmarks and is more robust to noisy ID data that may be misjudged as OOD data by other methods.

翻译：分布外（OOD）检测对于防止深度学习模型做出错误预测、确保人工智能系统安全至关重要。特别是在医疗诊断和自动驾驶等安全关键应用中，错误决策的代价通常难以承受。然而，神经网络常存在过度自信问题，即使对训练过程中从未见过且可能与训练数据（即分布内数据）无关的OOD数据，也会给出高置信度预测。确定预测的可靠性仍是一项艰巨且具有挑战性的任务。本文提出基于归一化逻辑的不确定性估计（UE-NL），一种鲁棒的OOD检测学习方法，具有三个主要优势：（1）采用UE-NL的神经网络通过预测输入数据的不确定性得分，在训练阶段将不确定性纳入softmax函数以调整难易样本的学习强度，从而对每个分布内样本进行同等处理，使模型实现鲁棒且准确的学习；（2）UE-NL对逻辑施加恒定向量范数，以消除优化过程中输出范数增长的影响（该影响在一定程度上导致了过度自信问题）；（3）UE-NL提供了一种新指标——不确定性得分的大小——用于检测OOD数据。实验表明，UE-NL在常见OOD基准测试中达到顶尖性能，且对可能被其他方法误判为OOD数据的含噪分布内数据具有更强鲁棒性。