Accurate neuronavigation is essential for generating the intended effect with transcranial magnetic stimulation (TMS). Precise coil placement also directly influences stimulation efficacy. Traditional neuronavigation systems often rely on costly and still hard to use and error-prone tracking systems. To solve these limitations, we present a computer-vision-based neuronavigation system for real-time tracking of patient and TMS instrumentation. The system can feed the necessary data for a digital twin to track TMS stimulation targets. We integrate a self-coordinating optical tracking system with multiple consumer-grade cameras and visible tags with a dynamic 3D brain model in Unity. This model updates in real time to represent the current stimulation coil position and the estimated stimulation point to intuitively visualize neural targets for clinicians. We incorporate an augmented reality (AR) module to bridge the gap between the visualization of the digital twin and the real world and project the brain model in real-time onto the head of a patient. AR headsets or mobile AR devices allow clinicians to interactively view and adjust the placement of the stimulation transducer intuitively instead of guidance through abstract numbers and 6D cross hairs on an external screen. The proposed technique provides improved spatial precision as well as accuracy. A case study with ten participants with a medical background also demonstrates that the system has high usability.

翻译：精确的神经导航对于经颅磁刺激产生预期效果至关重要。刺激线圈的精准放置也直接影响刺激疗效。传统的神经导航系统通常依赖于昂贵、使用困难且易出错的追踪系统。为解决这些限制，我们提出了一种基于计算机视觉的神经导航系统，用于实时追踪患者与TMS仪器。该系统可为数字孪生体提供必要数据以追踪TMS刺激目标。我们将一个自协调光学追踪系统（包含多台消费级相机与可见标记）与Unity中的动态3D脑模型相集成。该模型实时更新以呈现当前刺激线圈位置及预估刺激点，从而为临床医生直观可视化神经目标。我们引入增强现实模块以弥合数字孪生可视化与现实世界之间的差距，并将脑模型实时投影至患者头部。通过AR头显或移动AR设备，临床医生可直观交互式查看并调整刺激换能器的放置，而无需依赖外部屏幕上抽象数值与6D十字准线的引导。所提技术在空间精度与准确度方面均有提升。一项包含十名医学背景参与者的案例研究也表明该系统具有较高的可用性。