Tissue deformation poses a key challenge for accurate surgical scene reconstruction. Despite yielding high reconstruction quality, existing methods suffer from slow rendering speeds and long training times, limiting their intraoperative applicability. Motivated by recent progress in 3D Gaussian Splatting, an emerging technology in real-time 3D rendering, this work presents a novel fast reconstruction framework, termed Deform3DGS, for deformable tissues during endoscopic surgery. Specifically, we introduce 3D GS into surgical scenes by integrating a point cloud initialization to improve reconstruction. Furthermore, we propose a novel flexible deformation modeling scheme (FDM) to learn tissue deformation dynamics at the level of individual Gaussians. Our FDM can model the surface deformation with efficient representations, allowing for real-time rendering performance. More importantly, FDM significantly accelerates surgical scene reconstruction, demonstrating considerable clinical values, particularly in intraoperative settings where time efficiency is crucial. Experiments on DaVinci robotic surgery videos indicate the efficacy of our approach, showcasing superior reconstruction fidelity PSNR: (37.90) and rendering speed (338.8 FPS) while substantially reducing training time to only 1 minute/scene. Our code is available at https://github.com/jinlab-imvr/Deform3DGS.
翻译:组织变形是实现精确手术场景重建的关键挑战。尽管现有方法能够获得高质量的重建结果,但其渲染速度慢、训练时间长,限制了其在术中的实际应用。受实时三维渲染新兴技术——三维高斯泼溅(3D Gaussian Splatting)近期进展的启发,本研究提出了一种新颖的快速重建框架Deform3DGS,用于内窥镜手术中的可变形组织重建。具体而言,我们通过集成点云初始化将3D GS引入手术场景,以改进重建质量。此外,我们提出了一种新颖的柔性变形建模方案(FDM),用于在单个高斯层面学习组织变形动力学。我们的FDM能够以高效的表征方式对表面变形进行建模,从而实现实时渲染性能。更重要的是,FDM显著加速了手术场景的重建过程,展现出重要的临床价值,尤其是在时间效率至关重要的术中环境中。在达芬奇机器人手术视频上的实验验证了我们方法的有效性,其展示了卓越的重建保真度(PSNR: 37.90)和渲染速度(338.8 FPS),同时将训练时间大幅缩短至仅1分钟/场景。我们的代码公开于 https://github.com/jinlab-imvr/Deform3DGS。