Surgical interventions, particularly in neurology, represent complex and high-stakes scenarios that impose substantial cognitive burdens on surgical teams. Although deliberate education and practice can enhance cognitive capabilities, surgical training opportunities remain limited due to patient safety concerns. To address these cognitive challenges in surgical training and operation, we propose SurgBox, an agent-driven sandbox framework to systematically enhance the cognitive capabilities of surgeons in immersive surgical simulations. Specifically, our SurgBox leverages large language models (LLMs) with tailored Retrieval-Augmented Generation (RAG) to authentically replicate various surgical roles, enabling realistic training environments for deliberate practice. In particular, we devise Surgery Copilot, an AI-driven assistant to actively coordinate the surgical information stream and support clinical decision-making, thereby diminishing the cognitive workload of surgical teams during surgery. By incorporating a novel Long-Short Memory mechanism, our Surgery Copilot can effectively balance immediate procedural assistance with comprehensive surgical knowledge. Extensive experiments using real neurosurgical procedure records validate our SurgBox framework in both enhancing surgical cognitive capabilities and supporting clinical decision-making. By providing an integrated solution for training and operational support to address cognitive challenges, our SurgBox framework advances surgical education and practice, potentially transforming surgical outcomes and healthcare quality. The code is available at https://github.com/franciszchen/SurgBox.
翻译:外科手术干预,特别是在神经外科领域,代表着复杂且高风险的操作场景,给手术团队带来了巨大的认知负荷。尽管系统的教育与训练能够提升认知能力,但由于患者安全考虑,外科培训机会仍然有限。为应对外科培训与手术中的认知挑战,本文提出SurgBox——一种基于智能体驱动的沙箱框架,旨在通过沉浸式手术模拟系统性地提升外科医生的认知能力。具体而言,我们的SurgBox框架利用大语言模型(LLMs)结合定制化的检索增强生成(RAG)技术,真实复现多种手术角色,从而构建可供系统化训练的逼真手术环境。我们特别设计了手术协导系统(Surgery Copilot),这是一个基于人工智能的辅助系统,能够主动协调手术信息流并支持临床决策,从而降低手术团队在术中的认知负荷。通过引入新颖的长短期记忆机制,我们的手术协导系统能够有效平衡即时操作指导与全面手术知识支持。基于真实神经外科手术记录的大量实验验证了SurgBox框架在提升手术认知能力与支持临床决策方面的有效性。通过提供应对认知挑战的培训与操作支持一体化解决方案,我们的SurgBox框架推动了外科教育与临床实践的发展,有望改善手术结果与医疗质量。代码已开源:https://github.com/franciszchen/SurgBox。