Purpose: Tracking the 3D motion of the surgical tool and the patient anatomy is a fundamental requirement for computer-assisted skull-base surgery. The estimated motion can be used both for intra-operative guidance and for downstream skill analysis. Recovering such motion solely from surgical videos is desirable, as it is compliant with current clinical workflows and instrumentation. Methods: We present Tracker of Anatomy and Tool (TAToo). TAToo jointly tracks the rigid 3D motion of patient skull and surgical drill from stereo microscopic videos. TAToo estimates motion via an iterative optimization process in an end-to-end differentiable form. For robust tracking performance, TAToo adopts a probabilistic formulation and enforces geometric constraints on the object level. Results: We validate TAToo on both simulation data, where ground truth motion is available, as well as on anthropomorphic phantom data, where optical tracking provides a strong baseline. We report sub-millimeter and millimeter inter-frame tracking accuracy for skull and drill, respectively, with rotation errors below 1{\deg}. We further illustrate how TAToo may be used in a surgical navigation setting. Conclusion: We present TAToo, which simultaneously tracks the surgical tool and the patient anatomy in skull-base surgery. TAToo directly predicts the motion from surgical videos, without the need of any markers. Our results show that the performance of TAToo compares favorably to competing approaches. Future work will include fine-tuning of our depth network to reach a 1 mm clinical accuracy goal desired for surgical applications in the skull base.

翻译：目的：追踪手术器械与患者解剖结构的三维运动是计算机辅助颅底手术的基本需求。所估计的运动可用于术中导航和下游技能分析。从手术视频中仅依赖视觉信息恢复此类运动具有重要价值，因其与当前临床工作流程及设备无缝兼容。方法：我们提出解剖结构与器械追踪器（TAToo）。TAToo通过立体显微镜视频联合追踪患者颅骨与手术钻的刚体三维运动，采用端到端可微分的迭代优化过程估计运动。为实现稳健追踪性能，TAToo引入概率建模框架并在物体层面施加几何约束。结果：我们在模拟数据（可获得真实运动）和拟人模型数据（光学追踪提供强基线）上验证TAToo。颅骨与手术钻的帧间追踪精度分别达到亚毫米级和毫米级，旋转误差低于1{\deg}。我们进一步展示了TAToo在手术导航场景中的应用潜力。结论：我们提出TAToo，能同时追踪颅底手术中的手术器械与患者解剖结构。该方法可直接从手术视频预测运动，无需任何标记物。实验结果表明TAToo性能优于现有方法。未来工作将优化深度网络参数，以实现颅底手术所需的1毫米临床精度目标。