Visual anomaly detection aims to identify anomalous regions in images through unsupervised learning paradigms, with increasing application demand and value in fields such as industrial inspection and medical lesion detection. Despite significant progress in recent years, there is a lack of comprehensive benchmarks to adequately evaluate the performance of various mainstream methods across different datasets under the practical multi-class setting. The absence of standardized experimental setups can lead to potential biases in training epochs, resolution, and metric results, resulting in erroneous conclusions. This paper addresses this issue by proposing a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework that is highly extensible for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. Additionally, we have proposed the GPU-assisted ADEval package to address the slow evaluation problem of metrics like time-consuming mAU-PRO on large-scale data, significantly reducing evaluation time by more than \textit{1000-fold}. Through extensive experimental results, we objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection. We hope that ADer will become a valuable resource for researchers and practitioners in the field, promoting the development of more robust and generalizable anomaly detection systems. Full codes are open-sourced at https://github.com/zhangzjn/ader.

翻译：视觉异常检测旨在通过无监督学习范式识别图像中的异常区域，在工业检测和医学病灶检测等领域的应用需求与价值日益增长。尽管近年来取得显著进展，但在实际多类别设置下，仍缺乏能够全面评估不同数据集上各类主流方法性能的综合基准。标准化实验设置的缺失可能导致训练周期、分辨率和度量结果方面的潜在偏差，进而产生错误结论。本文通过提出一个综合性视觉异常检测基准ADer来解决该问题，该基准采用模块化框架设计，对新方法具有高度可扩展性。基准涵盖工业与医学领域的多个数据集，实现了十五种前沿方法和九项综合评估指标。此外，我们提出GPU加速的ADEval工具包，以解决大规模数据上耗时指标（如mAU-PRO）评估缓慢的问题，将评估时间显著降低超过\textit{1000倍}。通过大量实验结果，我们客观揭示了不同方法的优势与局限，并为多类别视觉异常检测的挑战与未来方向提供见解。我们希望ADer能成为该领域研究人员与实践者的宝贵资源，推动构建更鲁棒、更具泛化能力的异常检测系统。完整代码已开源：https://github.com/zhangzjn/ader。