Given a classifier, the inherent property of semantic Out-of-Distribution (OOD) samples is that their contents differ from all legal classes in terms of semantics, namely semantic mismatch. There is a recent work that directly applies it to OOD detection, which employs a conditional Generative Adversarial Network (cGAN) to enlarge semantic mismatch in the image space. While achieving remarkable OOD detection performance on small datasets, it is not applicable to ImageNet-scale datasets due to the difficulty in training cGANs with both input images and labels as conditions. As diffusion models are much easier to train and amenable to various conditions compared to cGANs, in this work, we propose to directly use pre-trained diffusion models for semantic mismatch-guided OOD detection, named DiffGuard. Specifically, given an OOD input image and the predicted label from the classifier, we try to enlarge the semantic difference between the reconstructed OOD image under these conditions and the original input image. We also present several test-time techniques to further strengthen such differences. Experimental results show that DiffGuard is effective on both Cifar-10 and hard cases of the large-scale ImageNet, and it can be easily combined with existing OOD detection techniques to achieve state-of-the-art OOD detection results.

翻译：摘要：对于给定的分类器，语义分布外样本的本质属性在于其内容在语义层面上与所有合法类别均存在差异，即语义失配。近期有研究将此特性直接应用于分布外检测，通过条件生成对抗网络在图像空间中放大语义失配。尽管该方法在小规模数据集上取得了显著的分布外检测性能，但由于需要同时以输入图像和标签作为条件训练条件生成对抗网络存在困难，因此无法适用于ImageNet规模的数据集。鉴于扩散模型相比条件生成对抗网络更易训练且能灵活适配多种条件，本文提出直接利用预训练扩散模型进行语义失配引导的分布外检测，命名为DiffGuard。具体而言，给定分布外输入图像及其分类器预测标签，我们尝试放大在此条件下重建的分布外图像与原始输入图像之间的语义差异。同时提出多种测试时增强技术进一步强化此类差异。实验结果表明，DiffGuard在Cifar-10以及大规模ImageNet的困难样本上均表现有效，且能便捷地结合现有分布外检测技术，实现最先进的分布外检测性能。