Despite the remarkable performance of video-based large language models (LLMs), their adversarial threat remains unexplored. To fill this gap, we propose the first adversarial attack tailored for video-based LLMs by crafting flow-based multi-modal adversarial perturbations on a small fraction of frames within a video, dubbed FMM-Attack. Extensive experiments show that our attack can effectively induce video-based LLMs to generate incorrect answers when videos are added with imperceptible adversarial perturbations. Intriguingly, our FMM-Attack can also induce garbling in the model output, prompting video-based LLMs to hallucinate. Overall, our observations inspire a further understanding of multi-modal robustness and safety-related feature alignment across different modalities, which is of great importance for various large multi-modal models. Our code is available at https://github.com/THU-Kingmin/FMM-Attack.

翻译：尽管基于视频的大语言模型（LLMs）性能卓越，但其对抗威胁尚未得到充分探索。为填补这一空白，我们提出了首个针对视频LLMs的对抗攻击方法——通过在视频中少量帧上构建基于流的多模态对抗扰动，称为FMM-Attack。大量实验表明，当视频被施加不易察觉的对抗扰动时，我们的攻击能够有效地诱导视频LLMs生成错误答案。值得注意的是，FMM-Attack还能引发模型输出紊乱，促使视频LLMs产生幻觉。总体而言，我们的观察结果促进了跨不同模态的多模态鲁棒性与安全相关特征对齐的进一步理解，这对各类大型多模态模型具有重要意义。我们的代码已开源至https://github.com/THU-Kingmin/FMM-Attack。