The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings. In this paper, we observe that the singular value distributions of the in-distribution (ID) and OOD features are quite different: the OOD feature matrix tends to have a larger dominant singular value than the ID feature, and the class predictions of OOD samples are largely determined by it. This observation motivates us to propose \texttt{RankFeat}, a simple yet effective \emph{post hoc} approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature. \texttt{RankFeat} achieves \emph{state-of-the-art} performance and reduces the average false positive rate (FPR95) by 17.90\% compared with the previous best method. The success of \texttt{RankFeat} motivates us to investigate whether a similar phenomenon would exist in the parameter matrices of neural networks. We thus propose \texttt{RankWeight} which removes the rank-1 weight from the parameter matrices of a single deep layer. Our \texttt{RankWeight}is also \emph{post hoc} and only requires computing the rank-1 matrix once. As a standalone approach, \texttt{RankWeight} has very competitive performance against other methods across various backbones. Moreover, \texttt{RankWeight} enjoys flexible compatibility with a wide range of OOD detection methods. The combination of \texttt{RankWeight} and \texttt{RankFeat} refreshes the new \emph{state-of-the-art} performance, achieving the FPR95 as low as 16.13\% on the ImageNet-1k benchmark. Extensive ablation studies and comprehensive theoretical analyses are presented to support the empirical results.

翻译：分布外（OOD）检测对于在现实场景中部署机器学习模型至关重要。本文观察到，分布内（ID）特征与OOD特征的奇异值分布存在显著差异：OOD特征矩阵的主导奇异值通常大于ID特征，且OOD样本的类别预测很大程度上由其决定。基于此发现，我们提出了一种简单有效的后处理方法RankFeat，通过从高层特征中移除由最大奇异值及其对应奇异向量构成的秩-1矩阵来实现OOD检测。RankFeat达到了最先进性能，与先前最优方法相比，将平均假阳性率（FPR95）降低了17.90%。受RankFeat成功的启发，我们进一步探究神经网络参数矩阵中是否存在类似现象，进而提出RankWeight方法——从单个深层参数矩阵中移除秩-1权重。RankWeight同样为后处理方法，仅需计算秩-1矩阵一次。作为独立方法，RankWeight在多种骨干网络上表现出极具竞争力的性能。此外，RankWeight能与广泛的OOD检测方法灵活兼容。将RankWeight与RankFeat结合，刷新了最先进性能，在ImageNet-1k基准上将FPR95降至16.13%。我们通过大量消融实验和全面理论分析支撑了实证结果。