Large Language Models (LLMs) can produce catastrophic responses in conversational settings that pose serious risks to public safety and security.Existing evaluations often fail to fully reveal these vulnerabilities because they rely on fixed attack prompt sequences, lack statistical guarantees, and do not scale to the vast space of multi-turn conversations.In this work, we propose C$^3$LLM, a novel, principled statistical Certification framework for Catastrophic risks in multi-turn Conversation for LLMs that bounds the probability of an LLM generating catastrophic responses under multi-turn conversation distributions with statistical guarantees.We model multi-turn conversations as probability distributions over query sequences, represented by a Markov process on a query graph whose edges encode semantic similarity to capture realistic conversational flow, and quantify catastrophic risks using confidence intervals. We define several inexpensive and practical distributions--random node, graph path, and adaptive with rejection. Our results demonstrate that these distributions can reveal substantial catastrophic risks in frontier models, with certified lower bounds as high as 70\% for the worst model, highlighting the urgent need for improved safety training strategies in frontier LLMs.

翻译：大型语言模型（LLMs）在对话场景中可能产生灾难性回复，对公共安全构成严重威胁。现有评估方法往往无法充分揭示这些脆弱性，因为它们依赖于固定的攻击提示序列、缺乏统计保证，且难以扩展到多轮对话的广阔空间。本研究提出C$^3$LLM——一种新颖的、基于原则的统计认证框架，用于量化LLMs在多轮对话中的灾难性风险。该框架能以统计保证的方式，界定LLM在多轮对话分布下生成灾难性回复的概率上限。我们将多轮对话建模为查询序列的概率分布，通过查询图上的马尔可夫过程表示，其中图的边编码语义相似性以捕捉真实的对话流，并利用置信区间量化灾难性风险。我们定义了几种计算成本低且实用的分布——随机节点分布、图路径分布和带拒绝的自适应分布。实验结果表明，这些分布能够揭示前沿模型中显著的灾难性风险，最差模型的认证风险下界高达70%，凸显了在前沿LLMs中改进安全训练策略的迫切需求。