Self-supervised learning in computer vision trains on unlabeled data, such as images or (image, text) pairs, to obtain an image encoder that learns high-quality embeddings for input data. Emerging backdoor attacks towards encoders expose crucial vulnerabilities of self-supervised learning, since downstream classifiers (even further trained on clean data) may inherit backdoor behaviors from encoders. Existing backdoor detection methods mainly focus on supervised learning settings and cannot handle pre-trained encoders especially when input labels are not available. In this paper, we propose DECREE, the first backdoor detection approach for pre-trained encoders, requiring neither classifier headers nor input labels. We evaluate DECREE on over 400 encoders trojaned under 3 paradigms. We show the effectiveness of our method on image encoders pre-trained on ImageNet and OpenAI's CLIP 400 million image-text pairs. Our method consistently has a high detection accuracy even if we have only limited or no access to the pre-training dataset.

翻译：自监督学习在计算机视觉领域通过未标注数据（如图像或(图像,文本)对）进行训练，获得能够为输入数据学习高质量嵌入的图像编码器。针对编码器的新型后门攻击揭示了自监督学习的关键漏洞，因为下游分类器（即使进一步在干净数据上训练）可能从编码器继承后门行为。现有后门检测方法主要关注监督学习场景，无法处理预训练编码器，尤其在输入标签不可用时。本文提出DECREE——首个面向预训练编码器的后门检测方法，该方法既不需要分类器头部，也不依赖输入标签。我们在超过400个通过三种范式植入木马的编码器上评估DECREE，展示了该方法在ImageNet预训练图像编码器及OpenAI的CLIP（4亿图像-文本对）上的有效性。即便我们仅能有限访问或无访问预训练数据集，该方法仍能保持高检测准确率。