Out-of-distribution (OOD) detection is essential in identifying test samples that deviate from the in-distribution (ID) data upon which a standard network is trained, ensuring network robustness and reliability. This paper introduces OOD knowledge distillation, a pioneering learning framework applicable whether or not training ID data is available, given a standard network. This framework harnesses OOD-sensitive knowledge from the standard network to craft a binary classifier adept at distinguishing between ID and OOD samples. To accomplish this, we introduce Confidence Amendment (CA), an innovative methodology that transforms an OOD sample into an ID one while progressively amending prediction confidence derived from the standard network. This approach enables the simultaneous synthesis of both ID and OOD samples, each accompanied by an adjusted prediction confidence, thereby facilitating the training of a binary classifier sensitive to OOD. Theoretical analysis provides bounds on the generalization error of the binary classifier, demonstrating the pivotal role of confidence amendment in enhancing OOD sensitivity. Extensive experiments spanning various datasets and network architectures confirm the efficacy of the proposed method in detecting OOD samples.

翻译：分布外（OOD）检测对于识别与标准网络训练所用的分布内（ID）数据存在偏差的测试样本至关重要，能确保网络的鲁棒性和可靠性。本文提出分布外知识蒸馏这一开创性学习框架，在给定标准网络的情况下，无论能否获取训练ID数据均可适用。该框架利用标准网络中蕴含的OOD敏感知识，构建一个能够有效区分ID与OOD样本的二分类器。为此，我们引入置信度修正（CA）创新方法，通过逐步修正标准网络输出的预测置信度，将OOD样本转化为ID样本。该方法可同时生成带有修正后预测置信度的ID与OOD样本，从而训练出对OOD敏感的判别分类器。理论分析给出了二分类器泛化误差的界，证明了置信度修正对提升OOD敏感性的关键作用。跨越多种数据集与网络架构的大量实验验证了所提方法在检测OOD样本方面的有效性。