Large Vision-Language Models (LVLMs) are vulnerable to a growing array of multimodal jailbreak attacks, necessitating defenses that are both generalizable to novel threats and efficient for practical deployment. Many current strategies fall short, either targeting specific attack patterns, which limits generalization, or imposing high computational overhead. While lightweight anomaly-detection methods offer a promising direction, we find that their common one-class design tends to confuse unseen benign inputs with malicious ones, leading to unreliable over-rejection. To address this, we propose Representational Contrastive Scoring (RCS), a framework built on a key insight: the most potent safety signals reside within the LVLM's own internal representations. Our approach inspects the internal geometry of these representations, learning a lightweight projection to maximally separate benign and malicious inputs in safety-critical layers. This enables a simple yet powerful contrastive score that differentiates true malicious intent from mere distribution shift. Our instantiations, MCD (Mahalanobis Contrastive Detection) and KCD (K-nearest Contrastive Detection), achieve state-of-the-art performance on a challenging evaluation protocol designed to test generalization to unseen attack types. This work demonstrates that effective jailbreak detection can be achieved by applying simple, interpretable statistical methods to the internal representations, offering a practical path towards safer LVLM deployment.

翻译：大型视觉-语言模型（LVLM）易受日益增多的多模态越狱攻击，亟需既能泛化至新型威胁又适合实际部署的高效防御策略。当前诸多方法存在局限：或针对特定攻击模式设计导致泛化能力不足，或引入高昂计算开销。尽管轻量级异常检测方法提供了有前景的方向，但我们发现其常见的单类设计易混淆未见过的良性输入与恶意输入，导致不可靠的过度拒绝。针对此问题，我们提出表征对比打分（RCS）框架，其核心洞察在于：最强效的安全信号蕴含于LVLM自身内部表征之中。该方法通过检测表征的内部几何结构，学习轻量级投影以在安全关键层中最大化分离良性输入与恶意输入，从而生成简洁而强大的对比分数，有效区分真实恶意意图与单纯分布偏移。我们的具体实现——马氏对比检测（MCD）与K近邻对比检测（KCD）——在专为测试对未见攻击类型泛化能力而设计的挑战性评估协议中达到最优性能。本研究证明，将简单可解释的统计方法应用于内部表征即可实现高效越狱检测，为更安全的LVLM部署提供了实用路径。