Few-shot Generalist Anomaly Detection requires models to generalize to novel categories without retraining, posing significant challenges in real-world scenarios with scarce samples and rapidly changing categories. Existing CLIP-based methods face two major challenges: coarse-grained unified text prompts struggle to adapt to fine-grained foreground-background differences, causing cross-granularity mismatch; and fine-tuning on auxiliary datasets disrupts CLIP's inherent open-world generalization due to domain shift, leading to cross-category generalization degradation. To address these, we propose to shift multimodal alignment entirely into a unified residual space, where residual representations naturally eliminate fine-grained normal feature differences across regions and class-specific biases, simultaneously resolving both problems. Based on this insight, Res$^2$CLIP, the first residual-to-residual alignment framework that symmetrically bridges visual and text modalities within CLIP's residual space, is designed. The framework is developed from a residual perspective into three branches: a text prompt-based branch, a visual prompt-based branch, and a novel residual-to-residual alignment branch. All learnable optimizations are constrained within the residual domain, and the residual alignment optimization objectives are designed to force the model to focus on relative anomaly deviations rather than optimizing class-specific features. Experiments on multiple datasets demonstrate the effectiveness of our architecture. The code is available at https://github.com/hito2448/Res2CLIP.

翻译：少样本通用异常检测要求模型无需重新训练即可泛化到新类别，这在样本稀缺且类别快速变化的真实场景中构成重大挑战。现有基于CLIP的方法面临两大难题：粗粒度统一文本提示难以适应细粒度前景-背景差异，导致跨粒度失配；在辅助数据集上微调会因领域偏移破坏CLIP固有的开放世界泛化能力，造成跨类别泛化退化。为此，我们提出将多模态对齐完全迁移到统一残差空间，其中残差表示天然消除了区域间细粒度正常特征差异与类别特定偏差，从而同时解决上述两个问题。基于这一洞察，我们设计了首个残差到残差对齐框架Res$^2$CLIP，该框架在CLIP残差空间内对称性地桥接视觉与文本模态。该框架从残差视角发展出三个分支：文本提示分支、视觉提示分支以及新型残差到残差对齐分支。所有可学习优化均约束在残差域内，且残差对齐优化目标旨在迫使模型关注相对异常偏差而非优化类别特定特征。在多个数据集上的实验证明了我们架构的有效性。代码发布于https://github.com/hito2448/Res2CLIP。