Integrated Speech and Large Language Models (SLMs) that can follow speech instructions and generate relevant text responses have gained popularity lately. However, the safety and robustness of these models remains largely unclear. In this work, we investigate the potential vulnerabilities of such instruction-following speech-language models to adversarial attacks and jailbreaking. Specifically, we design algorithms that can generate adversarial examples to jailbreak SLMs in both white-box and black-box attack settings without human involvement. Additionally, we propose countermeasures to thwart such jailbreaking attacks. Our models, trained on dialog data with speech instructions, achieve state-of-the-art performance on spoken question-answering task, scoring over 80% on both safety and helpfulness metrics. Despite safety guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs to adversarial perturbations and transfer attacks, with average attack success rates of 90% and 10% respectively when evaluated on a dataset of carefully designed harmful questions spanning 12 different toxic categories. However, we demonstrate that our proposed countermeasures reduce the attack success significantly.

翻译：集成语音与大型语言模型（SLMs）近年来因能遵循语音指令并生成相关文本响应而广受欢迎。然而，这些模型的安全性与鲁棒性仍不明确。本研究探索了此类指令遵循型语音语言模型在对抗攻击与越狱攻击中的潜在脆弱性。具体而言，我们设计了无需人工干预即可在白盒与黑盒攻击场景下生成对抗样本以越狱SLMs的算法。此外，我们提出了抵御此类越狱攻击的对策。我们的模型基于包含语音指令的对话数据训练，在口语问答任务中实现了最先进性能，安全性与有用性指标均超过80%。尽管存在安全防护措施，越狱实验仍表明SLMs易受对抗扰动与迁移攻击的影响——在涵盖12种有毒类别的精心设计有害问题数据集上，平均攻击成功率分别为90%与10%。然而，我们提出的对策能够显著降低攻击成功率。