Deploying large language models (LMs) can pose hazards from harmful outputs such as toxic or false text. Prior work has introduced automated tools that elicit harmful outputs to identify these risks. While this is a valuable step toward securing models, these approaches rely on a pre-existing way to efficiently classify undesirable outputs. Using a pre-existing classifier does not allow for red-teaming to be tailored to the target model. Furthermore, when failures can be easily classified in advance, red-teaming has limited marginal value because problems can be avoided by simply filtering training data and/or model outputs. Here, we consider red-teaming "from scratch," in which the adversary does not begin with a way to classify failures. Our framework consists of three steps: 1) Exploring the model's range of behaviors in the desired context; 2) Establishing a definition and measurement for undesired behavior (e.g., a classifier trained to reflect human evaluations); and 3) Exploiting the model's flaws using this measure to develop diverse adversarial prompts. We use this approach to red-team GPT-3 to discover classes of inputs that elicit false statements. In doing so, we construct the CommonClaim dataset of 20,000 statements labeled by humans as common-knowledge-true, common knowledge-false, or neither. We are making code and data available.

翻译：部署大型语言模型可能因生成有毒或虚假文本等有害输出而带来风险。先前工作引入了自动化工具来诱发有害输出以识别这些风险。尽管这是保障模型安全的重要一步，但这些方法依赖于预先存在的高效分类不良输出的手段。使用预定义分类器无法针对目标模型定制红队测试。此外，当失败可被提前轻松分类时，红队测试的边际价值有限，因为只需过滤训练数据和/或模型输出即可规避问题。本文考虑“从零开始”的红队测试，即攻击者最初不具备分类失败的方法。我们的框架包含三个步骤：1）探索模型在目标语境中的行为范围；2）建立不良行为的定义与测量方式（例如，训练反映人类评估的分类器）；3）利用该测量手段发掘模型缺陷，生成多样化的对抗性提示。我们采用该方法对GPT-3进行红队测试，以发现诱发虚假陈述的输入类别。在此过程中，我们构建了CommonClaim数据集，包含20,000条由人工标注为常识真、常识假或二者皆非的陈述。我们将公开代码与数据集。