Multimodal pretrained models are vulnerable to backdoor attacks, yet most existing methods rely on visual or multimodal triggers, which are impractical since visually embedded triggers rarely occur in real-world data. To overcome this limitation, we propose a novel Text-Guided Backdoor (TGB) attack on multimodal pretrained models, where commonly occurring words in textual descriptions serve as backdoor triggers, significantly improving stealthiness and practicality. Furthermore, we introduce visual adversarial perturbations on poisoned samples to modulate the model's learning of textual triggers, enabling a controllable and adjustable TGB attack. Extensive experiments on downstream tasks built upon multimodal pretrained models, including Composed Image Retrieval (CIR) and Visual Question Answering (VQA), demonstrate that TGB achieves practicality and stealthiness with adjustable attack success rates across diverse realistic settings, revealing critical security vulnerabilities in multimodal pretrained models.

翻译：多模态预训练模型极易受到后门攻击，然而现有方法大多依赖视觉或多模态触发器，由于视觉嵌入的触发器在真实世界数据中极少出现，导致其实用性不足。为突破这一局限，我们提出一种新颖的多模态预训练模型文本引导后门攻击方法（TGB），通过将文本描述中的常见词语作为后门触发器，显著提升了攻击的隐蔽性和实用性。此外，我们在带毒样本上引入视觉对抗扰动以调节模型对文本触发器的学习，从而实现对TGB攻击的可控可调。基于多模态预训练模型的下游任务（包括组合图像检索CIR和视觉问答VQA）中开展的大量实验表明，TGB能在多种现实场景下以可调节的攻击成功率实现实用性与隐蔽性并存，揭示了多模态预训练模型的关键安全漏洞。