3D neural implicit representations play a significant component in many robotic applications. However, reconstructing neural radiance fields (NeRF) from realistic event data remains a challenge due to the sparsities and the lack of information when only event streams are available. In this paper, we utilize motion, geometry, and density priors behind event data to impose strong physical constraints to augment NeRF training. The proposed novel pipeline can directly benefit from those priors to reconstruct 3D scenes without additional inputs. Moreover, we present a novel density-guided patch-based sampling strategy for robust and efficient learning, which not only accelerates training procedures but also conduces to expressions of local geometries. More importantly, we establish the first large dataset for event-based 3D reconstruction, which contains 101 objects with various materials and geometries, along with the groundtruth of images and depth maps for all camera viewpoints, which significantly facilitates other research in the related fields. The code and dataset will be publicly available at https://github.com/Mercerai/PAEv3d.

翻译：三维神经隐式表示在众多机器人应用中扮演着重要角色。然而，仅利用事件流数据时，由于数据的稀疏性和信息缺失，从真实事件数据中重建神经辐射场（NeRF）仍是一项挑战。本文利用事件数据背后的运动、几何和密度先验施加强物理约束，以增强NeRF训练。所提出的新型流水线可直接从这些先验中获益，在无需额外输入的情况下重建三维场景。此外，我们提出了一种密度引导的基于补丁的采样策略，以实现鲁棒且高效的学习，该策略不仅加速了训练过程，还有助于表达局部几何结构。更重要的是，我们建立了首个用于事件驱动三维重建的大规模数据集，包含101个具有不同材质和几何结构的物体，以及所有相机视角的图像和深度图真值，这将极大推动相关领域的研究。代码和数据集将在https://github.com/Mercerai/PAEv3d 公开提供。