Large language models (LLMs), such as ChatGPT, have achieved remarkable success across a wide range of fields. However, their trustworthiness remains a significant concern, as they are still susceptible to jailbreak attacks aimed at eliciting inappropriate or harmful responses. However, existing jailbreak attacks mainly operate at the natural language level and rely on a single attack strategy, limiting their effectiveness in comprehensively assessing LLM robustness. In this paper, we propose Equacode, a novel multi-strategy jailbreak approach for large language models via equation-solving and code completion. This approach transforms malicious intent into a mathematical problem and then requires the LLM to solve it using code, leveraging the complexity of cross-domain tasks to divert the model's focus toward task completion rather than safety constraints. Experimental results show that Equacode achieves an average success rate of 91.19% on the GPT series and 98.65% across 3 state-of-the-art LLMs, all with only a single query. Further, ablation experiments demonstrate that EquaCode outperforms either the mathematical equation module or the code module alone. This suggests a strong synergistic effect, thereby demonstrating that multi-strategy approach yields results greater than the sum of its parts.

翻译：大型语言模型（如ChatGPT）已在众多领域取得显著成功，但其可信度仍是一个重要关切，因为它们依然容易受到旨在诱导不当或有害回复的越狱攻击。然而，现有的越狱攻击主要在自然语言层面进行，且依赖单一攻击策略，限制了其在全面评估大型语言模型鲁棒性方面的有效性。本文提出EquaCode，一种通过方程求解与代码补全实现的新型大型语言模型多策略越狱方法。该方法将恶意意图转化为数学问题，然后要求大型语言模型使用代码进行求解，利用跨领域任务的复杂性将模型的注意力导向任务完成而非安全约束。实验结果表明，EquaCode在GPT系列模型上平均成功率达到91.19%，在3个最先进的大型语言模型上平均成功率达98.65%，且均仅需单次查询。此外，消融实验表明，EquaCode的表现优于单独使用数学方程模块或代码模块。这显示出强烈的协同效应，从而证明多策略方法能够产生超越各组成部分简单叠加的效果。