The rise of large language models (LLMs) has drawn attention to the existence of "jailbreaks" that allow the models to be used maliciously. However, there is no standard benchmark for measuring the severity of a jailbreak, leaving authors of jailbreak papers to create their own. We show that these benchmarks often include vague or unanswerable questions and use grading criteria that are biased towards overestimating the misuse potential of low-quality model responses. Some jailbreak techniques make the problem worse by decreasing the quality of model responses even on benign questions: we show that several jailbreaking techniques substantially reduce the zero-shot performance of GPT-4 on MMLU. Jailbreaks can also make it harder to elicit harmful responses from an "uncensored" open-source model. We present a new benchmark, StrongREJECT, which better discriminates between effective and ineffective jailbreaks by using a higher-quality question set and a more accurate response grading algorithm. We show that our new grading scheme better accords with human judgment of response quality and overall jailbreak effectiveness, especially on the sort of low-quality responses that contribute the most to over-estimation of jailbreak performance on existing benchmarks. We release our code and data at https://github.com/alexandrasouly/strongreject.

翻译：大语言模型的兴起引发了对其可被恶意利用的"越狱"现象的关注。然而，目前尚无衡量越狱严重程度的标准基准，导致越狱论文作者不得不自行设计评估方案。研究表明，这些基准通常包含模糊或无法回答的问题，并采用偏向高估低质量模型响应对误用潜在影响的评分标准。部分越狱技术甚至进一步恶化问题：即便面对良性问题，它们也会降低模型响应质量——我们证明多种越狱技术显著降低了GPT-4在MMLU基准上的零样本性能。越狱行为甚至会增加从"未经审查"的开源模型中诱导有害响应的难度。我们提出新基准StrongREJECT，通过采用更高质量的问题集和更精确的响应评分算法，更好地区分有效与无效越狱。实验表明，新的评分方案与人类对响应质量和越狱有效性的判断更为一致，尤其适用于那些最易导致现有基准高估越狱性能的低质量响应场景。我们的代码与数据已发布于https://github.com/alexandrasouly/strongreject。