Large Language Models (LLMs) demonstrate impressive zero-shot performance across a wide range of natural language processing tasks. Integrating various modality encoders further expands their capabilities, giving rise to Multimodal Large Language Models (MLLMs) that process not only text but also visual and auditory modality inputs. However, these advanced capabilities may also pose significant safety problems, as models can be exploited to generate harmful or inappropriate content through jailbreak attacks. While prior work has extensively explored how manipulating textual or visual modality inputs can circumvent safeguards in LLMs and MLLMs, the vulnerability of audio-specific jailbreak on Large Audio-Language Models (LALMs) remains largely underexplored. To address this gap, we introduce Jailbreak-AudioBench, which consists of the Toolbox, curated Dataset, and comprehensive Benchmark. The Toolbox supports not only text-to-audio conversion but also various editing techniques for injecting audio hidden semantics. The curated Dataset provides diverse explicit and implicit jailbreak audio examples in both original and edited forms. Utilizing this dataset, we evaluate multiple state-of-the-art LALMs and establish the most comprehensive Jailbreak benchmark to date for audio modality. Finally, Jailbreak-AudioBench establishes a foundation for advancing future research on LALMs safety alignment by enabling the in-depth exposure of more powerful jailbreak threats, such as query-based audio editing, and by facilitating the development of effective defense mechanisms.

翻译：大型语言模型（LLMs）在广泛的自然语言处理任务中展现出卓越的零样本性能。集成多种模态编码器进一步扩展了其能力，催生了能够处理文本、视觉及听觉模态输入的多模态大型语言模型（MLLMs）。然而，这些先进能力也可能引发严重的安全问题，因为模型可能通过越狱攻击被利用以生成有害或不恰当的内容。尽管先前研究已深入探讨了如何通过操纵文本或视觉模态输入来规避LLMs和MLLMs的安全防护，但针对大型音频语言模型（LALMs）的音频特异性越狱攻击的脆弱性仍未得到充分探索。为填补这一空白，我们提出了Jailbreak-AudioBench，该框架包含工具箱、精选数据集和综合基准测试。工具箱不仅支持文本到音频的转换，还提供多种用于注入音频隐藏语义的编辑技术。精选数据集以原始和编辑两种形式提供了多样化的显式与隐式越狱音频样本。利用该数据集，我们评估了多个最先进的LALMs，并建立了迄今为止音频模态领域最全面的越狱基准。最后，Jailbreak-AudioBench通过深度揭示更强大的越狱威胁（例如基于查询的音频编辑），并促进有效防御机制的开发，为推进未来LALMs安全对齐研究奠定了基础。