Contrastive Language-Image Pre-training models are widely reused across downstream interfaces, including feature extraction, retrieval, reranking, and selection. Existing CLIP backdoor, however, usually validate attacks on a small attack-native task, leaving unclear whether the same poisoned checkpoint remains exposed, weakens, or becomes not applicable when reused through other interfaces. We introduce DIFE, a Deployment-Interface Footprint Evaluation framework that audits backdoored CLIP checkpoints across deployment interfaces. DIFE makes various evaluations comparable by specifying each interface's component readout, trigger channel, target event, reference condition, and metric. DIFE also introduces effective-footprint diagnosis to identify the reusable CLIP component or component combination that carries exposure and explains where risk transfers. Auditing reproduced CLIP backdoors with DIFE reveals a structured landscape: native success is not a checkpoint-level risk certificate, exposure follows component footprints, text-side poisoning does not yield textual-encoder control, and some coupled attacks remain mechanism-bound. This audit reveals a import gapin existing CLIP backdoors: a textual encoder that itself becomes a reusable carrier of adversarial behavior. We therefore introduce BadTextTower to fill this gap. BadTextTower produces strong text-conditioned retrieval, reranking, and selection exposure while leaving visual-only reuse nearly clean.
翻译:对比语言-图像预训练模型被广泛重用于下游接口,包括特征提取、检索、重排序和选择。然而,现有的CLIP后门通常仅在小型攻击原生任务上验证攻击效果,导致同一中毒检查点通过其他接口重用时,其暴露程度是增强、减弱还是不可用尚不明确。我们提出DIFE(部署接口足迹评估框架),用于审计后门化CLIP检查点在各部署接口中的表现。DIFE通过定义每个接口的组件读取、触发通道、目标事件、参考条件和度量指标,使各种评估具有可比性。DIFE还引入有效足迹诊断,以识别承载暴露的可重用CLIP组件或组件组合,并解释风险转移的源头。利用DIFE审计复现的CLIP后门揭示了结构化的格局:原生成功并非检查点级别的风险证明,暴露遵循组件足迹,文本侧投毒不会产生文本编码器控制,某些耦合攻击仍受机制约束。该审计揭示了现有CLIP后门中的一个重要空白:文本编码器本身成为对抗行为的可重用载体。为此,我们提出BadTextTower填补这一空白。BadTextTower在产生强文本条件检索、重排序和选择暴露的同时,几乎不影响仅视觉重用的清洁性。