Anomaly detection requires detecting abnormal samples in large unlabeled datasets. While progress in deep learning and the advent of foundation models has produced powerful unsupervised anomaly detection methods, their deployment in practice is often hindered by the lack of labeled data -- without it, the detection accuracy of an anomaly detector cannot be evaluated reliably. In this work, we propose a general-purpose framework for evaluating image-based anomaly detectors with synthetically generated validation data. Our method assumes access to a small support set of normal images which are processed with a pre-trained diffusion model (our proposed method requires no training or fine-tuning) to produce synthetic anomalies. When mixed with normal samples from the support set, the synthetic anomalies create detection tasks that compose a validation framework for anomaly detection evaluation and model selection. In an extensive empirical study, ranging from natural images to industrial applications, we find that our synthetic validation framework selects the same models and hyper-parameters as selection with a ground-truth validation set. In addition, we find that prompts selected by our method for CLIP-based anomaly detection outperforms all other prompt selection strategies, and leads to the overall best detection accuracy, even on the challenging MVTec-AD dataset.

翻译：异常检测需要从大量未标注数据集中检测异常样本。尽管深度学习的发展和基础模型的出现催生了强大的无监督异常检测方法，但缺乏标注数据常阻碍其实际部署——若无标注数据，异常检测器的检测准确性无法被可靠评估。本研究提出一种通用框架，利用合成生成的验证数据评估基于图像的异常检测器。该方法假设可访问少量正常图像支持集，通过预训练扩散模型（无需额外训练或微调）处理这些正常图像以生成合成异常样本。当合成异常与支持集中的正常样本混合后，可构建检测任务组合，形成用于异常检测评估和模型选择的验证框架。从自然图像到工业应用的大规模实证研究表明，本合成验证框架能选择与真实验证集相同的模型和超参数。此外，采用本方法为基于CLIP的异常检测选择的提示词，其性能优于所有其他提示词选择策略，即使面对具有挑战性的MVTec-AD数据集，也能取得整体最优检测精度。