Endoscopy is essential in medical imaging, used for diagnosis, prognosis and treatment. Developing a robust dynamic 3D reconstruction pipeline for endoscopic videos could enhance visualization, improve diagnostic accuracy, aid in treatment planning, and guide surgery procedures. However, challenges arise due to the deformable nature of the tissues, the use of monocular cameras, illumination changes, occlusions and unknown camera trajectories. Inspired by neural rendering, we introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video. NeRFscopy includes a deformable model with a canonical radiance field and a time-dependent deformation field parameterized by SE(3) transformations. In addition, the color images are efficiently exploited by introducing sophisticated terms to learn a 3D implicit model without assuming any template or pre-trained model, solely from data. NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.

翻译：内窥镜成像是医学影像中不可或缺的技术，广泛应用于诊断、预后评估与治疗。为内窥镜视频开发鲁棒的动态三维重建流程，能够增强可视化效果、提升诊断准确性、辅助治疗规划并指导手术操作。然而，由于组织的可变形特性、单目相机的使用、光照变化、遮挡以及未知的相机轨迹，该任务面临诸多挑战。受神经渲染技术的启发，我们提出了NeRFscopy——一种基于单目视频实现可变形内窥镜组织新视角合成与三维重建的自监督流程。NeRFscopy包含一个由规范辐射场和基于SE(3)变换参数化的时变形变场构成的可变形模型。此外，通过引入精细化的约束项，系统能够高效利用彩色图像数据，在不依赖任何预设模板或预训练模型的情况下，仅从数据中学习三维隐式表示。NeRFscopy在新视角合成方面取得了精确的结果，在多种具有挑战性的内窥镜场景中均优于现有方法。