Complete reconstruction of surgical scenes is crucial for robot-assisted surgery (RAS). Deep depth estimation is promising but existing works struggle with depth discontinuities, resulting in noisy predictions at object boundaries and do not achieve complete reconstruction omitting occluded surfaces. To address these issues we propose EndoLRMGS, that combines Large Reconstruction Modelling (LRM) and Gaussian Splatting (GS), for complete surgical scene reconstruction. GS reconstructs deformable tissues and LRM generates 3D models for surgical tools while position and scale are subsequently optimized by introducing orthogonal perspective joint projection optimization (OPjPO) to enhance accuracy. In experiments on four surgical videos from three public datasets, our method improves the Intersection-over-union (IoU) of tool 3D models in 2D projections by>40%. Additionally, EndoLRMGS improves the PSNR of the tools projection from 3.82% to 11.07%. Tissue rendering quality also improves, with PSNR increasing from 0.46% to 49.87%, and SSIM from 1.53% to 29.21% across all test videos.

翻译：手术场景的完整重建对于机器人辅助手术至关重要。深度深度估计方法前景广阔，但现有工作难以处理深度不连续问题，导致物体边界处的预测结果存在噪声，且无法实现包含被遮挡表面的完整重建。为解决这些问题，我们提出EndoLRMGS，该方法结合大型重建建模与高斯溅射，以实现完整的手术场景重建。高斯溅射用于重建可变形组织，而大型重建建模则生成手术器械的三维模型；随后通过引入正交透视联合投影优化来优化其位置与尺度，以提升精度。在三个公开数据集的四段手术视频实验中，我们的方法将器械三维模型在二维投影中的交并比提升了超过40%。此外，EndoLRMGS将器械投影的峰值信噪比提升了3.82%至11.07%。组织渲染质量也得到改善，在所有测试视频中，峰值信噪比提升幅度为0.46%至49.87%，结构相似性指数提升幅度为1.53%至29.21%。