Large language models trained for safety and harmlessness remain susceptible to adversarial misuse, as evidenced by the prevalence of "jailbreak" attacks on early releases of ChatGPT that elicit undesired behavior. Going beyond recognition of the issue, we investigate why such attacks succeed and how they can be created. We hypothesize two failure modes of safety training: competing objectives and mismatched generalization. Competing objectives arise when a model's capabilities and safety goals conflict, while mismatched generalization occurs when safety training fails to generalize to a domain for which capabilities exist. We use these failure modes to guide jailbreak design and then evaluate state-of-the-art models, including OpenAI's GPT-4 and Anthropic's Claude v1.3, against both existing and newly designed attacks. We find that vulnerabilities persist despite the extensive red-teaming and safety-training efforts behind these models. Notably, new attacks utilizing our failure modes succeed on every prompt in a collection of unsafe requests from the models' red-teaming evaluation sets and outperform existing ad hoc jailbreaks. Our analysis emphasizes the need for safety-capability parity -- that safety mechanisms should be as sophisticated as the underlying model -- and argues against the idea that scaling alone can resolve these safety failure modes.

翻译：摘要：尽管经过安全与无害性训练，大型语言模型仍易受到对抗性滥用的影响，例如ChatGPT早期版本中诱发不良行为的“越狱”攻击屡见不鲜。在识别该问题之外，我们进一步探究了此类攻击为何成功以及如何构建。我们假设安全训练存在两种失效模式：目标冲突与泛化失配。目标冲突指模型的能力与安全目标存在矛盾，而泛化失配则指安全训练无法泛化到已具备能力的领域。我们利用这些失效模式指导越狱设计，并针对现有及新设计的攻击评估了最新模型（包括OpenAI的GPT-4和Anthropic的Claude v1.3）。研究发现，尽管这些模型背后经历了广泛的红队测试与安全训练，漏洞依然存在。值得注意的是，基于我们提出的失效模式的新攻击，在模型红队评估集中所有不安全请求提示上均成功实施，且性能优于现有临时性越狱方法。我们的分析强调了安全-能力对等性的必要性——即安全机制需与底层模型同样精妙——并反对仅靠扩展规模即可解决这些安全失效模式的观点。