Novel view synthesis from low dynamic range (LDR) blurry images, which are common in the wild, struggles to recover high dynamic range (HDR) and sharp 3D representations in extreme lighting conditions. Although existing methods employ event data to address this issue, they ignore the sensor-physics mismatches between the camera output and physical world radiance, resulting in suboptimal HDR and deblurring results. To cope with this problem, we propose a unified sensor-physics grounded NeRF framework for sharp HDR novel view synthesis from single-exposure blurry LDR images and corresponding events. We employ NeRF to directly represent the actual radiance of the 3D scene in the HDR domain and model raw HDR scene rays hitting the sensor pixels as in the physical world. A pixel-wise RGB mapping field is introduced to align the above rendered pixel values with the sensor-recorded LDR pixel values of the input images. A novel event mapping field is also designed to bridge the physical scene dynamics and actual event sensor output. The two mapping fields are jointly optimized with the NeRF network, leveraging the spatial and temporal dynamic information in events to enhance the sharp HDR 3D representation learning. Experiments on the collected and public datasets demonstrate that our method can achieve state-of-the-art deblurring HDR novel view synthesis results with single-exposure blurry LDR images and corresponding events.

翻译：从低动态范围模糊图像中进行新视角合成，这类图像在真实场景中普遍存在，在极端光照条件下难以恢复高动态范围与清晰的三维表征。尽管现有方法利用事件数据应对此问题，但它们忽略了相机输出与物理世界辐亮度之间的传感器物理失配，导致高动态范围重建与去模糊效果欠佳。为解决此问题，我们提出一种基于传感器物理的统一NeRF框架，能够从单曝光模糊低动态范围图像及对应事件数据中合成清晰的高动态范围新视角。我们采用NeRF直接在HDR域中表征三维场景的真实辐亮度，并模拟物理世界中传感器像素接收原始HDR场景光线的过程。通过引入像素级RGB映射场，将上述渲染像素值与输入图像中传感器记录的低动态范围像素值对齐。同时设计了一种新颖的事件映射场，以衔接物理场景动态特性与实际事件传感器输出。这两个映射场与NeRF网络联合优化，利用事件数据中的时空动态信息增强清晰高动态范围三维表征的学习。在自采集与公开数据集上的实验表明，我们的方法能够通过单曝光模糊低动态范围图像及对应事件数据，实现当前最优的去模糊高动态范围新视角合成效果。