Large Multimodal Models (LMMs) have demonstrated the ability to interact with humans under real-world conditions by combining Large Language Models (LLMs) and modality encoders to align multimodal information (visual and auditory) with text. However, such models raise new safety challenges of whether models that are safety-aligned on text also exhibit consistent safeguards for multimodal inputs. Despite recent safety-alignment research on vision LMMs, the safety of audio LMMs remains under-explored. In this work, we comprehensively red team the safety of five advanced audio LMMs under three settings: (i) harmful questions in both audio and text formats, (ii) harmful questions in text format accompanied by distracting non-speech audio, and (iii) speech-specific jailbreaks. Our results under these settings demonstrate that open-source audio LMMs suffer an average attack success rate of 69.14% on harmful audio questions, and exhibit safety vulnerabilities when distracted with non-speech audio noise. Our speech-specific jailbreaks on Gemini-1.5-Pro achieve an attack success rate of 70.67% on the harmful query benchmark. We provide insights on what could cause these reported safety-misalignments. Warning: this paper contains offensive examples.

翻译：大型多模态模型通过结合大型语言模型与模态编码器，将多模态信息（视觉与听觉）与文本对齐，已展现出在真实世界条件下与人类交互的能力。然而，此类模型也引发了新的安全挑战：在文本上已进行安全对齐的模型，是否对多模态输入也表现出同等的安全防护？尽管近期已有针对视觉大型多模态模型的安全对齐研究，但音频大型多模态模型的安全性仍未得到充分探索。在本工作中，我们系统地对五种先进的音频大型多模态模型在三种设定下进行了红队安全测试：（i）以音频和文本格式呈现的有害问题，（ii）以文本格式呈现的有害问题并辅以分散注意力的非语音音频，以及（iii）针对语音的特定越狱攻击。我们在这些设定下的测试结果表明，开源音频大型多模态模型在有害音频问题上的平均攻击成功率高达69.14%，并且在受到非语音音频噪声干扰时表现出安全脆弱性。我们在Gemini-1.5-Pro上实施的语音特定越狱攻击，在有害查询基准上达到了70.67%的攻击成功率。我们进一步分析了可能导致这些安全失准现象的原因。警告：本文包含具有冒犯性的示例。