Reconstructing deformable tissues from endoscopic videos is essential in many downstream surgical applications. However, existing methods suffer from slow rendering speed, greatly limiting their practical use. In this paper, we introduce EndoGaussian, a real-time endoscopic scene reconstruction framework built on 3D Gaussian Splatting (3DGS). By integrating the efficient Gaussian representation and highly-optimized rendering engine, our framework significantly boosts the rendering speed to a real-time level. To adapt 3DGS for endoscopic scenes, we propose two strategies, Holistic Gaussian Initialization (HGI) and Spatio-temporal Gaussian Tracking (SGT), to handle the non-trivial Gaussian initialization and tissue deformation problems, respectively. In HGI, we leverage recent depth estimation models to predict depth maps of input binocular/monocular image sequences, based on which pixels are re-projected and combined for holistic initialization. In SPT, we propose to model surface dynamics using a deformation field, which is composed of an efficient encoding voxel and a lightweight deformation decoder, allowing for Gaussian tracking with minor training and rendering burden. Experiments on public datasets demonstrate our efficacy against prior SOTAs in many aspects, including better rendering speed (195 FPS real-time, 100$\times$ gain), better rendering quality (37.848 PSNR), and less training overhead (within 2 min/scene), showing significant promise for intraoperative surgery applications. Code is available at: \url{https://yifliu3.github.io/EndoGaussian/}.

翻译：从内窥镜视频中重建可变形组织对于许多下游外科应用至关重要。然而，现有方法渲染速度缓慢，极大限制了其实用性。本文提出EndoGaussian——一种基于三维高斯泼溅（3DGS）的实时内窥镜场景重建框架。通过融合高效的高斯表示与高度优化的渲染引擎，该框架将渲染速度显著提升至实时水平。为使3DGS适应内窥镜场景，我们提出整体高斯初始化（HGI）与时空高斯追踪（SGT）两种策略，分别处理非平凡的高斯初始化与组织变形问题。HGI策略利用近期深度估计模型预测输入双目/单目图像序列的深度图，基于此将像素重投影并融合以实现整体初始化。SGT策略则提出采用由高效编码体素与轻量级变形解码器组成的变形场来建模表面动力学，使得高斯追踪仅需极小的训练与渲染开销。在公开数据集上的实验表明，本方法在渲染速度（实时195 FPS，提升100倍）、渲染质量（PSNR达37.848）及训练开销（每场景<2分钟）等多方面优于先前最优方法，展现出术中手术应用的显著潜力。代码地址：\url{https://yifliu3.github.io/EndoGaussian/}。