Depth estimation plays a crucial role in various tasks within endoscopic surgery, including navigation, surface reconstruction, and augmented reality visualization. Despite the significant achievements of foundation models in vision tasks, including depth estimation, their direct application to the medical domain often results in suboptimal performance. This highlights the need for efficient adaptation methods to adapt these models to endoscopic depth estimation. We propose Endoscopic Depth Any Camera (EndoDAC) which is an efficient self-supervised depth estimation framework that adapts foundation models to endoscopic scenes. Specifically, we develop the Dynamic Vector-Based Low-Rank Adaptation (DV-LoRA) and employ Convolutional Neck blocks to tailor the foundational model to the surgical domain, utilizing remarkably few trainable parameters. Given that camera information is not always accessible, we also introduce a self-supervised adaptation strategy that estimates camera intrinsics using the pose encoder. Our framework is capable of being trained solely on monocular surgical videos from any camera, ensuring minimal training costs. Experiments demonstrate that our approach obtains superior performance even with fewer training epochs and unaware of the ground truth camera intrinsics. Code is available at https://github.com/BeileiCui/EndoDAC.
翻译:深度估计在内窥镜手术的诸多任务中扮演着关键角色,包括导航、表面重建和增强现实可视化。尽管基础模型在视觉任务(包括深度估计)中取得了显著成就,但其在医学领域的直接应用往往导致性能欠佳。这凸显了开发高效适配方法以将此类模型应用于内窥镜深度估计的必要性。我们提出内窥镜任意相机深度估计(EndoDAC)框架——一种高效的自监督深度估计框架,可将基础模型适配至内窥镜场景。具体而言,我们开发了动态向量基低秩适配(DV-LoRA)方法,并采用卷积颈模块对基础模型进行外科领域定制化调整,仅使用极少量可训练参数。考虑到相机参数并非始终可获取,我们还引入了一种自监督适配策略,利用位姿编码器估计相机内参。该框架可仅使用来自任意相机的单目手术视频进行训练,从而确保最低训练成本。实验结果表明,即便使用更少的训练轮次且未知真实相机内参,本方法仍能获得卓越性能。代码已开源至 https://github.com/BeileiCui/EndoDAC。