Clinical decisions are often required under incomplete information. Clinical experts must identify whether available information is sufficient for judgment, as both premature conclusion and unnecessary abstention can compromise patient safety. To evaluate this capability of large language models (LLMs), we developed ClinDet-Bench, a benchmark based on clinical scoring systems that decomposes incomplete-information scenarios into determinable and undeterminable conditions. Identifying determinability requires considering all hypotheses about missing information, including unlikely ones, and verifying whether the conclusion holds across them. We find that recent LLMs fail to identify determinability under incomplete information, producing both premature judgments and excessive abstention, despite correctly explaining the underlying scoring knowledge and performing well under complete information. These findings suggest that existing benchmarks are insufficient to evaluate the safety of LLMs in clinical settings. ClinDet-Bench provides a framework for evaluating determinability recognition, leading to appropriate abstention, with potential applicability to medicine and other high-stakes domains, and is publicly available.

翻译：临床决策常常需要在信息不完整的情况下进行。临床专家必须判断现有信息是否足以做出结论，因为过早下定论和不必要的弃权都可能危及患者安全。为了评估大语言模型（LLMs）的这项能力，我们开发了ClinDet-Bench，这是一个基于临床评分系统的基准测试，它将信息不完整场景分解为可确定和不可确定两种条件。识别可确定性需要考虑关于缺失信息的所有假设（包括可能性较低的假设），并验证结论是否在所有假设下均成立。我们发现，尽管近期的大语言模型能够正确解释基础评分知识，并在信息完整时表现良好，但在信息不完整的情况下却无法识别可确定性，既会产生过早的判断，也会出现过度弃权。这些发现表明，现有基准测试不足以评估大语言模型在临床环境中的安全性。ClinDet-Bench提供了一个用于评估可确定性识别能力的框架，从而促成适当的弃权，该框架在医学及其他高风险领域具有潜在适用性，并已公开发布。