Online health resources and large language models (LLMs) are increasingly used as a first point of contact for medical decision-making, yet their reliability in healthcare remains limited by low accuracy, lack of transparency, and susceptibility to unverified information. We introduce a proof-of-concept conversational self-triage system that guides LLMs with 100 clinically validated flowcharts from the American Medical Association, providing a structured and auditable framework for patient decision support. The system leverages a multi-agent framework consisting of a retrieval agent, a decision agent, and a chat agent to identify the most relevant flowchart, interpret patient responses, and deliver personalized, patient-friendly recommendations, respectively. Performance was evaluated at scale using synthetic datasets of simulated conversations. The system achieved 95.29% top-3 accuracy in flowchart retrieval (N=2,000) and 99.10% accuracy in flowchart navigation across varied conversational styles and conditions (N=37,200). By combining the flexibility of free-text interaction with the rigor of standardized clinical protocols, this approach demonstrates the feasibility of transparent, accurate, and generalizable AI-assisted self-triage, with potential to support informed patient decision-making while improving healthcare resource utilization.
翻译:在线健康资源和大型语言模型(LLM)日益成为医疗决策的初始接触点,但其在医疗保健领域的可靠性仍受限于准确性低、缺乏透明度以及易受未经验证信息的影响。我们提出了一种概念验证型对话式自诊分流系统,该系统利用美国医学会提供的100份经过临床验证的流程图来引导LLM,为患者决策支持提供了一个结构化且可审计的框架。该系统采用多智能体框架,包含检索智能体、决策智能体和对话智能体,分别负责识别最相关的流程图、解读患者应答并提供个性化的、便于患者理解的建议。通过使用模拟对话的合成数据集进行了大规模性能评估。在流程图检索任务中(N=2,000),系统实现了95.29%的Top-3准确率;在多样化的对话风格和条件下(N=37,200),流程图导航准确率达到99.10%。通过将自由文本交互的灵活性与标准化临床规程的严谨性相结合,该方法展示了透明、准确且可推广的AI辅助自诊分流的可行性,有望在支持患者知情决策的同时,提升医疗资源的利用效率。