Out-of-distribution detection (OOD) is a crucial technique for deploying machine learning models in the real world to handle the unseen scenarios. In this paper, we first propose a simple yet effective Neural Activation Prior (NAP) for OOD detection. Our neural activation prior is based on a key observation that, for a channel before the global pooling layer of a fully trained neural network, the probability of a few neurons being activated with a large response by an in-distribution (ID) sample is significantly higher than that by an OOD sample. An intuitive explanation is that for a model fully trained on ID dataset, each channel would play a role in detecting a certain pattern in the ID dataset, and a few neurons can be activated with a large response when the pattern is detected in an input sample. Then, a new scoring function based on this prior is proposed to highlight the role of these strongly activated neurons in OOD detection. Our approach is plug-and-play and does not lead to any performance degradation on ID data classification and requires no extra training or statistics from training or external datasets. Notice that previous methods primarily rely on post-global-pooling features of the neural networks, while the within-channel distribution information we leverage would be discarded by the global pooling operator. Consequently, our method is orthogonal to existing approaches and can be effectively combined with them in various applications. Experimental results show that our method achieves the state-of-the-art performance on CIFAR benchmark and ImageNet dataset, which demonstrates the power of the proposed prior. Finally, we extend our method to Transformers and the experimental findings indicate that NAP can also significantly enhance the performance of OOD detection on Transformers, thereby demonstrating the broad applicability of this prior knowledge.

翻译：分布外检测（OOD）是将机器学习模型部署到现实世界以处理未见场景的关键技术。本文首先提出了一种简单而有效的神经激活先验（NAP）用于OOD检测。我们的神经激活先验基于一个关键观察：对于一个经过充分训练的神经网络在全局池化层之前的通道，由分布内（ID）样本激活少数神经元并产生大响应的概率显著高于OOD样本。一个直观的解释是，对于在ID数据集上充分训练的模型，每个通道在检测ID数据集中特定模式时会发挥作用，当输入样本中检测到该模式时，少数神经元会被激活并产生大响应。随后，我们基于该先验提出了一种新的评分函数，以突出这些强激活神经元在OOD检测中的作用。我们的方法即插即用，不会导致ID数据分类性能下降，且无需额外训练或来自训练集/外部数据集的统计信息。值得注意的是，先前的方法主要依赖神经网络的全局池化后特征，而我们所利用的通道内分布信息会被全局池化算子丢弃。因此，我们的方法与现有方法正交，并可在各种应用中与它们有效结合。实验结果表明，我们的方法在CIFAR基准测试和ImageNet数据集上达到了最先进的性能，这证明了所提先验的有效性。最后，我们将方法扩展到Transformer架构，实验结果表明NAP也能显著提升Transformer的OOD检测性能，从而证明了该先验知识的广泛适用性。