The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i.e., in-distribution (InD). In this paper, we study the OOD problem from a neuron activation view. We first formulate neuron activation states by considering both the neuron output and its influence on model decisions. Then, to characterize the relationship between neurons and OOD issues, we introduce the \textit{neuron activation coverage} (NAC) -- a simple measure for neuron behaviors under InD data. Leveraging our NAC, we show that 1) InD and OOD inputs can be largely separated based on the neuron behavior, which significantly eases the OOD detection problem and beats the 21 previous methods over three benchmarks (CIFAR-10, CIFAR-100, and ImageNet-1K). 2) a positive correlation between NAC and model generalization ability consistently holds across architectures and datasets, which enables a NAC-based criterion for evaluating model robustness. Compared to prevalent InD validation criteria, we show that NAC not only can select more robust models, but also has a stronger correlation with OOD test performance.

翻译：分布外（OOD）问题通常出现在神经网络遇到显著偏离训练数据分布（即分布内，InD）的数据时。本文从神经元激活视角研究OOD问题。我们首先综合考虑神经元输出及其对模型决策的影响，形式化定义了神经元激活状态。随后，为刻画神经元与OOD问题之间的关系，我们引入\textit{神经元激活覆盖}（Neuron Activation Coverage，NAC）——一种衡量InD数据下神经元行为的简单指标。利用NAC，我们证明：1）基于神经元行为可大幅分离InD与OOD输入，这显著简化了OOD检测问题，并在三个基准数据集（CIFAR-10、CIFAR-100和ImageNet-1K）上超越了21种现有方法；2）NAC与模型泛化能力之间存在跨架构与数据集的稳定正相关性，由此可建立基于NAC的模型鲁棒性评估准则。相较于当前主流的InD验证准则，NAC不仅能筛选出更鲁棒的模型，而且与OOD测试性能的相关性更强。