Auditing large language models for unexpected behaviors is critical to preempt catastrophic deployments, yet remains challenging. In this work, we cast auditing as an optimization problem, where we automatically search for input-output pairs that match a desired target behavior. For example, we might aim to find a non-toxic input that starts with "Barack Obama" that a model maps to a toxic output. This optimization problem is difficult to solve as the set of feasible points is sparse, the space is discrete, and the language models we audit are non-linear and high-dimensional. To combat these challenges, we introduce a discrete optimization algorithm, ARCA, that jointly and efficiently optimizes over inputs and outputs. Our approach automatically uncovers derogatory completions about celebrities (e.g. "Barack Obama is a legalized unborn" -> "child murderer"), produces French inputs that complete to English outputs, and finds inputs that generate a specific name. Our work offers a promising new tool to uncover models' failure-modes before deployment.

翻译：审计大型语言模型中的意外行为对于预防灾难性部署至关重要，但仍具有挑战性。在本工作中，我们将审计问题转化为一个优化问题，自动搜索与期望目标行为匹配的输入-输出对。例如，我们可能旨在寻找一个以"Barack Obama"开头的非毒性输入，使得模型输出毒性内容。该优化问题求解困难，因为可行点集稀疏、搜索空间离散，且被审计的语言模型具有非线性和高维特性。为应对这些挑战，我们提出了一种离散优化算法ARCA，该算法能够联合且高效地优化输入和输出。我们的方法能自动发现关于名人的贬义续写（如"Barack Obama is a legalized unborn" -> "child murderer"），生成输出英文的法语输入，以及找到可生成特定名称的输入。本工作为预先揭示模型部署前的故障模式提供了一种富有前景的新工具。