Current state-of-the-art multi-class unsupervised anomaly detection (MUAD) methods rely on training encoder--decoder models to reconstruct anomaly-free features. However, we argue that such task-specific training is costly under distribution shifts, and that reconstruction-based residual scoring further faces a fidelity--stability dilemma. Existing training-free alternatives, in turn, remain prone to cross-category and cross-region mismatches in MUAD. Motivated by these limitations, we propose Retrieval-based Anomaly Detection (RAD), a task-specific training-free framework that stores anomaly-free features in a memory and detects anomalies through multi-level retrieval, matching test patches against the memory. Experiments demonstrate that RAD achieves state-of-the-art performance across four established benchmarks (MVTec-AD, VisA, Real-IAD, 3D-ADAM) under both standard and few-shot settings. On MVTec-AD, RAD reaches 96.7% Pixel AUROC with just a single anomaly-free image compared to 98.5% of RAD's full-data performance. Collectively, these findings overturn the assumption that MUAD requires task-specific training, showing that state-of-the-art anomaly detection is feasible with training-free memory-based retrieval. Our code is available at https://github.com/longkukuhi/RAD.

翻译：当前最先进的多类无监督异常检测（MUAD）方法依赖于训练编码器-解码器模型以重建无异常特征。然而，我们认为此类任务特定训练在分布偏移下成本高昂，且基于重建的残差评分进一步面临保真度-稳定性困境。而现有免训练替代方法在多类无监督异常检测中仍易出现跨类别和跨区域不匹配问题。受上述局限启发，我们提出基于检索的异常检测（RAD），这是一个无需任务特定训练的框架：将无异常特征存储于记忆库中，通过多级检索匹配测试块与记忆库来检测异常。实验表明，在标准设置和少样本设置下，RAD在四个公认基准（MVTec-AD、VisA、Real-IAD、3D-ADAM）上均达到最先进性能。在MVTec-AD上，RAD仅使用单张无异常图像即可达到96.7%的像素级AUROC，而全数据性能为98.5%。综合来看，这些发现推翻了MUAD需要任务特定训练的假设，证明了基于免训练记忆检索的异常检测可实现最先进水平。我们的代码已开源：https://github.com/longkukuhi/RAD。