Neural Radiance Fields (NeRFs) have shown great potential in novel view synthesis. However, they struggle to render sharp images when the data used for training is affected by motion blur. On the other hand, event cameras excel in dynamic scenes as they measure brightness changes with microsecond resolution and are thus only marginally affected by blur. Recent methods attempt to enhance NeRF reconstructions under camera motion by fusing frames and events. However, they face challenges in recovering accurate color content or constrain the NeRF to a set of predefined camera poses, harming reconstruction quality in challenging conditions. This paper proposes a novel formulation addressing these issues by leveraging both model- and learning-based modules. We explicitly model the blur formation process, exploiting the event double integral as an additional model-based prior. Additionally, we model the event-pixel response using an end-to-end learnable response function, allowing our method to adapt to non-idealities in the real event-camera sensor. We show, on synthetic and real data, that the proposed approach outperforms existing deblur NeRFs that use only frames as well as those that combine frames and events by +6.13dB and +2.48dB, respectively.

翻译：神经辐射场（NeRFs）在新视角合成中展现出巨大潜力。然而，当训练数据受运动模糊影响时，其难以渲染出清晰的图像。另一方面，事件相机在动态场景中表现优异，因其能以微秒级分辨率测量亮度变化，因而几乎不受模糊影响。现有方法尝试通过融合帧数据与事件数据来改善相机运动下的NeRF重建效果，但它们在恢复准确色彩内容方面存在困难，或将NeRF约束于一组预定义的相机位姿，从而在复杂条件下损害重建质量。本文提出一种新颖的建模方法，通过结合模型驱动与学习驱动的模块来解决上述问题。我们显式建模模糊形成过程，利用事件数据的双重积分作为额外的模型先验。同时，我们采用端到端可学习的响应函数来建模事件-像素响应，使方法能够适应真实事件相机传感器中的非理想特性。在合成数据与真实数据上的实验表明，所提方法在仅使用帧数据的去模糊NeRF方法上提升+6.13dB，在结合帧与事件的方法上提升+2.48dB，均优于现有技术。