One of the most critical factors in achieving sharp Novel View Synthesis (NVS) using neural field methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) is the quality of the training images. However, Conventional RGB cameras are susceptible to motion blur. In contrast, neuromorphic cameras like event and spike cameras inherently capture more comprehensive temporal information, which can provide a sharp representation of the scene as additional training data. Recent methods have explored the integration of event cameras to improve the quality of NVS. The event-RGB approaches have some limitations, such as high training costs and the inability to work effectively in the background. Instead, our study introduces a new method that uses the spike camera to overcome these limitations. By considering texture reconstruction from spike streams as ground truth, we design the Texture from Spike (TfS) loss. Since the spike camera relies on temporal integration instead of temporal differentiation used by event cameras, our proposed TfS loss maintains manageable training costs. It handles foreground objects with backgrounds simultaneously. We also provide a real-world dataset captured with our spike-RGB camera system to facilitate future research endeavors. We conduct extensive experiments using synthetic and real-world datasets to demonstrate that our design can enhance novel view synthesis across NeRF and 3DGS. The code and dataset will be made available for public access.

翻译：实现基于神经辐射场（NeRF）和三维高斯泼溅（3DGS）等神经场方法的清晰新视角合成（NVS）的关键因素之一是训练图像的质量。然而，传统RGB相机易受运动模糊影响。相比之下，事件相机和脉冲相机等神经形态相机能够固有地捕获更全面的时间信息，从而提供场景的清晰表示作为额外训练数据。近期研究探索了集成事件相机以提升NVS质量的方法。事件-RGB方法存在训练成本高、背景区域效果不佳等局限性。相反，本研究提出一种利用脉冲相机克服这些局限的新方法。通过将脉冲流生成的纹理重建视为真值，我们设计了纹理从脉冲（TfS）损失函数。由于脉冲相机依赖于时间积分（而非事件相机使用的时间差分），我们提出的TfS损失函数保持了可管理的训练成本，并能同时处理前景物体与背景。此外，我们还提供了一个使用脉冲-RGB相机系统捕获的真实世界数据集，以促进未来研究工作。我们基于合成数据和真实世界数据集进行了广泛实验，证明所提设计能够增强基于NeRF和3DGS的新视角合成效果。代码与数据集将公开发布。