Endoscopic procedures are crucial for colorectal cancer diagnosis, and three-dimensional reconstruction of the environment for real-time novel-view synthesis can significantly enhance diagnosis. We present PR-ENDO, a framework that leverages 3D Gaussian Splatting within a physically based, relightable model tailored for the complex acquisition conditions in endoscopy, such as restricted camera rotations and strong view-dependent illumination. By exploiting the connection between the camera and light source, our approach introduces a relighting model to capture the intricate interactions between light and tissue using physically based rendering and MLP. Existing methods often produce artifacts and inconsistencies under these conditions, which PR-ENDO overcomes by incorporating a specialized diffuse MLP that utilizes light angles and normal vectors, achieving stable reconstructions even with limited training camera rotations. We benchmarked our framework using a publicly available dataset and a newly introduced dataset with wider camera rotations. Our methods demonstrated superior image quality compared to baseline approaches.

翻译：内窥镜检查对于结直肠癌的诊断至关重要，而针对环境进行三维重建以实现实时新视角合成，能显著提升诊断效果。我们提出了PR-ENDO，这是一个在内窥镜复杂采集条件下（如受限的相机旋转和强烈的视角相关光照）量身定制的、基于物理的可重光照模型中利用3D高斯溅射的框架。通过利用相机与光源之间的关联，我们的方法引入了一个重光照模型，该模型结合基于物理的渲染和MLP来捕捉光线与组织之间复杂的相互作用。现有方法在这些条件下常常产生伪影和不一致性，PR-ENDO通过引入一个利用光线角度和法向量的专用漫反射MLP来克服这些问题，即使在训练相机旋转有限的情况下也能实现稳定的重建。我们使用一个公开可用的数据集和一个新引入的、具有更宽相机旋转范围的数据集对我们的框架进行了基准测试。与基线方法相比，我们的方法展现了更优的图像质量。