Conversational AI is starting to support real clinical work, but most evaluation methods miss how compliance depends on the full course of a conversation. We introduce Obligatory-Information Phase Structured Compliance Evaluation (OIP-SCE), an evaluation method that checks whether every required clinical obligation is met, in the right order, with clear evidence for clinicians to review. This makes complex rules practical and auditable, helping close the gap between technical progress and what healthcare actually needs. We demonstrate the method in two case studies (respiratory history, benefits verification) and show how phase-level evidence turns policy into shared, actionable steps. By giving clinicians control over what to check and engineers a clear specification to implement, OIP-SCE provides a single, auditable evaluation surface that aligns AI capability with clinical workflow and supports routine, safe use.
翻译:会话式人工智能正开始支持真实的临床工作,但大多数评估方法忽略了合规性如何依赖于对话的完整过程。我们引入了必需信息阶段结构化合规性评估(OIP-SCE),这是一种评估方法,用于检查每项必需的临床义务是否以正确的顺序得到满足,并为临床医生提供清晰的证据以供审查。这使得复杂规则变得实用且可审计,有助于缩小技术进步与医疗实际需求之间的差距。我们在两个案例研究(呼吸系统病史、福利验证)中展示了该方法,并说明了阶段级证据如何将政策转化为共享、可操作的步骤。通过让临床医生控制检查内容,并为工程师提供清晰的实施规范,OIP-SCE提供了一个单一、可审计的评估界面,使AI能力与临床工作流程保持一致,并支持常规、安全的使用。