Learning neural implicit representations has achieved remarkable performance in 3D reconstruction from multi-view images. Current methods use volume rendering to render implicit representations into either RGB or depth images that are supervised by multi-view ground truth. However, rendering a view each time suffers from incomplete depth at holes and unawareness of occluded structures from the depth supervision, which severely affects the accuracy of geometry inference via volume rendering. To resolve this issue, we propose to learn neural implicit representations from multi-view RGBD images through volume rendering with an attentive depth fusion prior. Our prior allows neural networks to perceive coarse 3D structures from the Truncated Signed Distance Function (TSDF) fused from all depth images available for rendering. The TSDF enables accessing the missing depth at holes on one depth image and the occluded parts that are invisible from the current view. By introducing a novel attention mechanism, we allow neural networks to directly use the depth fusion prior with the inferred occupancy as the learned implicit function. Our attention mechanism works with either a one-time fused TSDF that represents a whole scene or an incrementally fused TSDF that represents a partial scene in the context of Simultaneous Localization and Mapping (SLAM). Our evaluations on widely used benchmarks including synthetic and real-world scans show our superiority over the latest neural implicit methods. Project page: https://machineperceptionlab.github.io/Attentive_DF_Prior/

翻译：从多视角图像学习神经隐式表示在三维重建中取得了显著性能。现有方法通过体渲染将隐式表示渲染为RGB图像或深度图像，并利用多视角真实值进行监督。然而，每次渲染单个视图时，深度图空洞处存在不完整深度，且深度监督无法感知遮挡结构，严重影响了体积渲染中几何推断的准确性。为解决该问题，我们提出通过具有注意力深度融合先验的体积渲染，从多视角RGBD图像学习神经隐式表示。该先验使神经网络能够从所有可用于渲染的深度图像融合得到的截断符号距离函数中感知粗粒度三维结构。TSDF能够获取单个深度图空洞处的缺失深度以及当前视角不可见的遮挡部分。通过引入新型注意力机制，我们允许神经网络将推断的占据度作为学习到的隐式函数，直接利用深度融合先验。该注意力机制可同时适用于表示完整场景的一次性融合TSDF，以及同时定位与建图场景中表示部分场景的增量融合TSDF。在包含合成数据与真实扫描数据的广泛基准测试上的评估表明，我们的方法优于最新的神经隐式方法。项目页面：https://machineperceptionlab.github.io/Attentive_DF_Prior/