One of the most critical factors in achieving sharp Novel View Synthesis (NVS) using neural field methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) is the quality of the training images. However, Conventional RGB cameras are susceptible to motion blur. In contrast, neuromorphic cameras like event and spike cameras inherently capture more comprehensive temporal information, which can provide a sharp representation of the scene as additional training data. Recent methods have explored the integration of event cameras to improve the quality of NVS. The event-RGB approaches have some limitations, such as high training costs and the inability to work effectively in the background. Instead, our study introduces a new method that uses the spike camera to overcome these limitations. By considering texture reconstruction from spike streams as ground truth, we design the Texture from Spike (TfS) loss. Since the spike camera relies on temporal integration instead of temporal differentiation used by event cameras, our proposed TfS loss maintains manageable training costs. It handles foreground objects with backgrounds simultaneously. We also provide a real-world dataset captured with our spike-RGB camera system to facilitate future research endeavors. We conduct extensive experiments using synthetic and real-world datasets to demonstrate that our design can enhance novel view synthesis across NeRF and 3DGS. The code and dataset will be made available for public access.

翻译：实现基于神经场方法（如NeRF和3D高斯泼溅）的清晰新视角合成的关键因素之一是训练图像的质量。然而，传统RGB相机易受运动模糊影响。相比之下，事件相机和脉冲相机等神经形态相机能够天然捕获更全面的时域信息，从而提供场景的清晰表征作为额外训练数据。近期研究已探索通过集成事件相机提升NVS质量，但事件-RGB方法存在训练成本高、无法有效处理背景等局限性。本研究提出一种利用脉冲相机克服这些局限的新方法：通过将脉冲流纹理重建视为真值，设计"纹理从脉冲"损失。由于脉冲相机基于时域积分（而非事件相机采用的时域微分），所提TfS损失在保持可控训练成本的同时，能同步处理前景物体与背景。我们同时提供基于自研脉冲-RGB相机系统采集的真实世界数据集，以促进后续研究。通过在合成与真实数据集上的大量实验表明，本设计方案可增强NeRF与3DGS的新视角合成效果。代码与数据集将面向公众开放。