As a neuromorphic sensor with high temporal resolution, spike cameras offer notable advantages over traditional cameras in high-speed vision applications such as high-speed optical estimation, depth estimation, and object tracking. Inspired by the success of the spike camera, we proposed Spike-NeRF, the first Neural Radiance Field derived from spike data, to achieve 3D reconstruction and novel viewpoint synthesis of high-speed scenes. Instead of the multi-view images at the same time of NeRF, the inputs of Spike-NeRF are continuous spike streams captured by a moving spike camera in a very short time. To reconstruct a correct and stable 3D scene from high-frequency but unstable spike data, we devised spike masks along with a distinctive loss function. We evaluate our method qualitatively and numerically on several challenging synthetic scenes generated by blender with the spike camera simulator. Our results demonstrate that Spike-NeRF produces more visually appealing results than the existing methods and the baseline we proposed in high-speed scenes. Our code and data will be released soon.

翻译：作为一种具有高时间分辨率的神经形态传感器，脉冲相机相较于传统相机在高速视觉应用（如高速光流估计、深度估计和目标跟踪）中展现出显著优势。受脉冲相机成功应用的启发，我们提出了Spike-NeRF——首个基于脉冲数据的神经辐射场，旨在实现高速场景的三维重建与新视角合成。与NeRF依赖同一时刻的多视角图像不同，Spike-NeRF的输入是运动脉冲相机在极短时间内采集的连续脉冲流。为从高频但不稳定的脉冲数据中重建正确且稳定的三维场景，我们设计了脉冲掩码以及独特的损失函数。我们通过Blender结合脉冲相机模拟器生成的多个具有挑战性的合成场景，对所提方法进行了定性和定量评估。结果表明，在高速场景下，Spike-NeRF比现有方法及我们提出的基线方法能生成更美观的视觉效果。相关代码与数据将很快公开。