Large Language Models (LLMs) can produce catastrophic responses in conversational settings that pose serious risks to public safety and security. Existing evaluations often fail to fully reveal these vulnerabilities because they rely on fixed attack prompt sequences, lack statistical guarantees, and do not scale to the vast space of multi-turn conversations. In this work, we propose C$^3$LLM, a novel, principled statistical Certification framework for Catastrophic risks in multi-turn Conversation for LLMs that bounds the probability of an LLM generating catastrophic responses under multi-turn conversation distributions with statistical guarantees. We model multi-turn conversations as probability distributions over query sequences, represented by a Markov process on a query graph whose edges encode semantic similarity to capture realistic conversational flow, and quantify catastrophic risks using confidence intervals. We define several inexpensive and practical distributions--random node, graph path, and adaptive with rejection. Our results demonstrate that these distributions can reveal substantial catastrophic risks in frontier models, with certified lower bounds as high as 70% for the worst model, highlighting the urgent need for improved safety training strategies in frontier LLMs.

翻译：大型语言模型（LLMs）在对话场景中可能产生具有灾难性后果的响应，对公共安全构成严重威胁。现有评估方法往往无法充分揭示这些脆弱性，因为它们依赖固定的攻击提示序列、缺乏统计保证，且难以扩展到多轮对话的广阔空间。本研究提出C$^3$LLM——一个针对LLMs多轮对话中灾难性风险的创新性统计认证框架，该框架能以统计保证的方式界定LLM在多轮对话分布下生成灾难性响应的概率边界。我们将多轮对话建模为查询序列的概率分布，通过查询图上的马尔可夫过程实现表征，其中边编码语义相似性以捕捉真实对话流，并利用置信区间量化灾难性风险。我们定义了几种计算成本低廉且实用的分布——随机节点分布、图路径分布以及带拒绝的自适应分布。实验结果表明，这些分布能有效揭示前沿模型中存在的显著灾难性风险，最差模型的认证风险下限高达70%，这凸显了在前沿LLMs中改进安全训练策略的迫切需求。