Jailbreak attacks cause large language models (LLMs) to generate harmful, unethical, or otherwise objectionable content. Evaluating these attacks presents a number of challenges, which the current collection of benchmarks and evaluation techniques do not adequately address. First, there is no clear standard of practice regarding jailbreaking evaluation. Second, existing works compute costs and success rates in incomparable ways. And third, numerous works are not reproducible, as they withhold adversarial prompts, involve closed-source code, or rely on evolving proprietary APIs. To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (2) a jailbreaking dataset comprising 100 behaviors -- both original and sourced from prior work -- which align with OpenAI's usage policies; (3) a standardized evaluation framework at https://github.com/JailbreakBench/jailbreakbench that includes a clearly defined threat model, system prompts, chat templates, and scoring functions; and (4) a leaderboard at https://jailbreakbench.github.io/ that tracks the performance of attacks and defenses for various LLMs. We have carefully considered the potential ethical implications of releasing this benchmark, and believe that it will be a net positive for the community.

翻译：越狱攻击导致大型语言模型生成有害、不道德或其他令人反感的内容。评估这些攻击存在诸多挑战，而当前的基准测试集合与评估技术未能充分应对这些挑战。首先，关于越狱评估尚无明确的标准实践。其次，现有工作以不可比较的方式计算成本与成功率。再者，许多研究因保留对抗性提示、涉及闭源代码或依赖不断演变的专有API而无法复现。为应对这些挑战，我们推出JailbreakBench，这是一个包含以下组件的开源基准：(1) 一个持续更新的最先进对抗性提示库（我们称之为越狱工件）；(2) 一个包含100种行为（既有原创行为，也有源自先前工作的行为）的越狱数据集，这些行为均符合OpenAI的使用政策；(3) 一个位于 https://github.com/JailbreakBench/jailbreakbench 的标准化评估框架，包含明确定义的威胁模型、系统提示、聊天模板和评分函数；(4) 一个位于 https://jailbreakbench.github.io/ 的排行榜，用于追踪不同大型语言模型的攻击与防御性能。我们已仔细考量发布此基准可能带来的伦理影响，并相信它将为研究社区带来积极的净效益。