Asynchronously operating event cameras find many applications due to their high dynamic range, vanishingly low motion blur, low latency and low data bandwidth. The field saw remarkable progress during the last few years, and existing event-based 3D reconstruction approaches recover sparse point clouds of the scene. However, such sparsity is a limiting factor in many cases, especially in computer vision and graphics, that has not been addressed satisfactorily so far. Accordingly, this paper proposes the first approach for 3D-consistent, dense and photorealistic novel view synthesis using just a single colour event stream as input. At its core is a neural radiance field trained entirely in a self-supervised manner from events while preserving the original resolution of the colour event channels. Next, our ray sampling strategy is tailored to events and allows for data-efficient training. At test, our method produces results in the RGB space at unprecedented quality. We evaluate our method qualitatively and numerically on several challenging synthetic and real scenes and show that it produces significantly denser and more visually appealing renderings than the existing methods. We also demonstrate robustness in challenging scenarios with fast motion and under low lighting conditions. We release the newly recorded dataset and our source code to facilitate the research field, see https://4dqv.mpi-inf.mpg.de/EventNeRF.

翻译：异步运行的事件相机因其高动态范围、极低的运动模糊、低延迟和低数据带宽而在众多应用中得到应用。近年来，该领域取得了显著进展，现有基于事件的三维重建方法可恢复场景的稀疏点云。然而，这种稀疏性在许多情况下（尤其是计算机视觉和图形学领域）成为限制因素，且迄今未得到令人满意的解决。为此，本文提出首个仅以单色事件流为输入、实现三维一致、密集且逼真的新视角合成方法。其核心是一个完全以自监督方式从事件中训练的神经辐射场，同时保留了彩色事件通道的原始分辨率。此外，我们提出的射线采样策略专为事件设计，可实现数据高效的训练。在测试阶段，我们的方法能在RGB空间中生成前所未有的高质量结果。我们在多个具有挑战性的合成场景和真实场景上进行了定性和定量评估，结果表明该方法比现有方法能生成更密集、更具视觉吸引力的渲染图像。我们还演示了在快速运动和弱光照条件下的鲁棒性。为促进该研究领域，我们发布了新录制的数据集和源代码，参见 https://4dqv.mpi-inf.mpg.de/EventNeRF。