The security concerns surrounding Large Language Models (LLMs) have been extensively explored, yet the safety of Multimodal Large Language Models (MLLMs) remains understudied. In this paper, we observe that Multimodal Large Language Models (MLLMs) can be easily compromised by query-relevant images, as if the text query itself were malicious. To address this, we introduce MM-SafetyBench, a comprehensive framework designed for conducting safety-critical evaluations of MLLMs against such image-based manipulations. We have compiled a dataset comprising 13 scenarios, resulting in a total of 5,040 text-image pairs. Our analysis across 12 state-of-the-art models reveals that MLLMs are susceptible to breaches instigated by our approach, even when the equipped LLMs have been safety-aligned. In response, we propose a straightforward yet effective prompting strategy to enhance the resilience of MLLMs against these types of attacks. Our work underscores the need for a concerted effort to strengthen and enhance the safety measures of open-source MLLMs against potential malicious exploits. The resource is available at https://github.com/isXinLiu/MM-SafetyBench

翻译：大型语言模型（LLMs）的安全问题已得到广泛探讨，然而多模态大语言模型（MLLMs）的安全性研究仍显不足。本文发现，多模态大语言模型（MLLMs）极易受到与查询相关图像的干扰，其效果犹如文本查询本身具有恶意性。为此，我们提出了MM-SafetyBench——一个针对此类基于图像操控进行MLLMs安全关键评估的综合性框架。我们构建了涵盖13类场景的数据集，共计5,040个文本-图像对。通过对12个前沿模型的分析，我们发现即使配备的LLMs已进行安全对齐，MLLMs仍易受我们方法所引发的安全漏洞影响。对此，我们提出了一种简洁而有效的提示策略，以增强MLLMs抵御此类攻击的能力。本研究强调需要协同努力，以加强和提升开源MLLMs应对潜在恶意利用的安全防护措施。相关资源已发布于https://github.com/isXinLiu/MM-SafetyBench。