Reconstructing a 3D surface from colonoscopy video is challenging due to illumination and reflectivity variation in the video frame that can cause defective shape predictions. Aiming to overcome this challenge, we utilize the characteristics of surface normal vectors and develop a two-step neural framework that significantly improves the colonoscopy reconstruction quality. The normal-based depth initialization network trained with self-supervised normal consistency loss provides depth map initialization to the normal-depth refinement module, which utilizes the relationship between illumination and surface normals to refine the frame-wise normal and depth predictions recursively. Our framework's depth accuracy performance on phantom colonoscopy data demonstrates the value of exploiting the surface normals in colonoscopy reconstruction, especially on en face views. Due to its low depth error, the prediction result from our framework will require limited post-processing to be clinically applicable for real-time colonoscopy reconstruction.

翻译：从结肠镜视频中重建三维表面具有挑战性，因为视频帧中的光照和反射率变化可能导致形状预测出现缺陷。为克服这一挑战，我们利用表面法向量的特性，开发了一个两步式神经框架，显著提升了结肠镜重建质量。采用自监督法线一致性损失训练的基于法线的深度初始化网络，为法线-深度优化模块提供深度图初始化，该模块利用光照与表面法线之间的关系，递归优化逐帧法线和深度预测。本框架在体模结肠镜数据上的深度精度表现，证明了在结肠镜重建中利用表面法线的价值，尤其对于正面视图。由于其低深度误差，本框架的预测结果只需有限的后期处理，即可应用于临床实时结肠镜重建。