Large language models (LLMs) present a valuable technology for various applications in healthcare, but their tendency to hallucinate introduces unacceptable uncertainty in critical decision-making situations. Human-AI collaboration (HAIC) can mitigate this uncertainty by combining human and AI strengths for better outcomes. This paper presents a novel guided deferral system that provides intelligent guidance when AI defers cases to human decision-makers. We leverage LLMs' verbalisation capabilities and internal states to create this system, demonstrating that fine-tuning smaller LLMs with data from larger models enhances performance while maintaining computational efficiency. A pilot study showcases the effectiveness of our deferral system.

翻译：大语言模型为医疗领域的多种应用提供了有价值的技术，但其产生幻觉的倾向在关键决策场景中引入了不可接受的不确定性。人机协作能够通过结合人类与人工智能的优势来缓解这种不确定性，从而获得更优结果。本文提出了一种新颖的引导式委托系统，当人工智能将病例委托给人类决策者时，该系统可提供智能指导。我们利用大语言模型的言语化能力与内部状态构建该系统，证明了使用大模型数据微调较小的大语言模型能够在保持计算效率的同时提升性能。一项初步研究展示了我们委托系统的有效性。