Given a classifier, the inherent property of semantic Out-of-Distribution (OOD) samples is that their contents differ from all legal classes in terms of semantics, namely semantic mismatch. There is a recent work that directly applies it to OOD detection, which employs a conditional Generative Adversarial Network (cGAN) to enlarge semantic mismatch in the image space. While achieving remarkable OOD detection performance on small datasets, it is not applicable to ImageNet-scale datasets due to the difficulty in training cGANs with both input images and labels as conditions. As diffusion models are much easier to train and amenable to various conditions compared to cGANs, in this work, we propose to directly use pre-trained diffusion models for semantic mismatch-guided OOD detection, named DiffGuard. Specifically, given an OOD input image and the predicted label from the classifier, we try to enlarge the semantic difference between the reconstructed OOD image under these conditions and the original input image. We also present several test-time techniques to further strengthen such differences. Experimental results show that DiffGuard is effective on both Cifar-10 and hard cases of the large-scale ImageNet, and it can be easily combined with existing OOD detection techniques to achieve state-of-the-art OOD detection results.

翻译：摘要：给定一个分类器，语义分布外样本的固有属性是其内容在语义上与所有合法类别不同，即语义不匹配。近期有一项工作直接将其应用于分布外检测，通过使用条件生成对抗网络（cGAN）在图像空间中放大语义不匹配。尽管该方法在小型数据集上取得了显著的分布外检测性能，但由于训练同时以输入图像和标签为条件的cGAN存在困难，因而难以适用于ImageNet规模的数据集。鉴于扩散模型相比cGAN更易训练且能适应多种条件，本文提出直接使用预训练扩散模型进行语义不匹配引导的分布外检测，命名为DiffGuard。具体而言，给定一个分布外输入图像及分类器预测的标签，我们尝试放大在这些条件下重建的分布外图像与原始输入图像之间的语义差异。我们还提出了若干测试时技术以进一步增强此类差异。实验结果表明，DiffGuard在Cifar-10及大规模ImageNet的困难案例上均有效，且可轻松与现有分布外检测技术结合，实现最先进的分布外检测结果。