Safety is critical to the usage of large language models (LLMs). Multiple techniques such as data filtering and supervised fine-tuning have been developed to strengthen LLM safety. However, currently known techniques presume that corpora used for safety alignment of LLMs are solely interpreted by semantics. This assumption, however, does not hold in real-world applications, which leads to severe vulnerabilities in LLMs. For example, users of forums often use ASCII art, a form of text-based art, to convey image information. In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics. We show that five SOTA LLMs (GPT-3.5, GPT-4, Gemini, Claude, and Llama2) struggle to recognize prompts provided in the form of ASCII art. Based on this observation, we develop the jailbreak attack ArtPrompt, which leverages the poor performance of LLMs in recognizing ASCII art to bypass safety measures and elicit undesired behaviors from LLMs. ArtPrompt only requires black-box access to the victim LLMs, making it a practical attack. We evaluate ArtPrompt on five SOTA LLMs, and show that ArtPrompt can effectively and efficiently induce undesired behaviors from all five LLMs.

翻译：安全对于大型语言模型（LLMs）的使用至关重要。目前已开发出多种技术（如数据过滤和监督微调）来增强LLM的安全性。然而，现有技术均假设用于LLM安全对齐的语料库仅能通过语义进行解释。这一假设在实际应用中并不成立，从而导致LLM存在严重漏洞。例如，论坛用户常使用ASCII艺术（一种基于文本的艺术形式）来传达图像信息。本文提出了一种新颖的基于ASCII艺术的越狱攻击，并引入了全面的基准测试——视觉文本挑战（Vision-in-Text Challenge, ViTC），以评估LLM识别无法仅通过语义解释的提示的能力。研究表明，五种最先进的LLM（GPT-3.5、GPT-4、Gemini、Claude和Llama2）均难以识别以ASCII艺术形式呈现的提示。基于此发现，我们开发了越狱攻击ArtPrompt，利用LLM在识别ASCII艺术方面的性能缺陷来绕过安全措施，从而诱导LLM产生不良行为。ArtPrompt仅需对受害者LLM进行黑盒访问，具有实际攻击性。我们在五种最先进的LLM上评估了ArtPrompt，结果表明ArtPrompt能有效且高效地诱导所有五种LLM产生不良行为。