Learning neural implicit representations has achieved remarkable performance in 3D reconstruction from multi-view images. Current methods use volume rendering to render implicit representations into either RGB or depth images that are supervised by multi-view ground truth. However, rendering a view each time suffers from incomplete depth at holes and unawareness of occluded structures from the depth supervision, which severely affects the accuracy of geometry inference via volume rendering. To resolve this issue, we propose to learn neural implicit representations from multi-view RGBD images through volume rendering with an attentive depth fusion prior. Our prior allows neural networks to perceive coarse 3D structures from the Truncated Signed Distance Function (TSDF) fused from all depth images available for rendering. The TSDF enables accessing the missing depth at holes on one depth image and the occluded parts that are invisible from the current view. By introducing a novel attention mechanism, we allow neural networks to directly use the depth fusion prior with the inferred occupancy as the learned implicit function. Our attention mechanism works with either a one-time fused TSDF that represents a whole scene or an incrementally fused TSDF that represents a partial scene in the context of Simultaneous Localization and Mapping (SLAM). Our evaluations on widely used benchmarks including synthetic and real-world scans show our superiority over the latest neural implicit methods. Project page: https://machineperceptionlab.github.io/Attentive_DF_Prior/

翻译：从多视角图像中进行三维重建时，学习神经隐式表示已取得显著成效。现有方法通过体渲染将隐式表示渲染为受多视角真值监督的RGB图像或深度图像。然而，每次渲染单个视角时，深度空洞处存在缺失深度，且无法感知被遮挡结构，这严重影响了基于体渲染的几何推理精度。为解决此问题，我们提出通过引入注意力深度融合先验的体渲染，从多视角RGBD图像中学习神经隐式表示。该先验允许神经网络从所有可用于渲染的深度图像融合成的截断符号距离函数（TSDF）中感知粗粒度三维结构。TSDF能够获取单张深度图像上空洞处的缺失深度，以及当前视角不可见的被遮挡部分。通过引入新型注意力机制，我们使神经网络能够直接将深度融合先验与推断出的占用率作为学习到的隐式函数。该注意力机制既支持表示整个场景的一次性融合TSDF，也支持在即时定位与地图构建（SLAM）场景下表示局部场景的增量融合TSDF。我们在广泛使用的合成与真实扫描基准数据集上的评估表明，本方法优于最新神经隐式方法。项目页面：https://machineperceptionlab.github.io/Attentive_DF_Prior/