High-quality imaging of dynamic scenes in extremely low-light conditions is highly challenging. Photon scarcity induces severe noise and texture loss, causing significant image degradation. Event cameras, featuring a high dynamic range (120 dB) and high sensitivity to motion, serve as powerful complements to conventional cameras by offering crucial cues for preserving subtle textures. However, most existing approaches emphasize texture recovery from events, while paying little attention to image noise or the intrinsic noise of events themselves, which ultimately hinders accurate pixel reconstruction under photon-starved conditions. In this work, we propose NEC-Diff, a novel diffusion-based event-RAW hybrid imaging framework that extracts reliable information from heavily noisy signals to reconstruct fine scene structures. The framework is driven by two key insights: (1) combining the linear light-response property of RAW images with the brightness-change nature of events to establish a physics-driven constraint for robust dual-modal denoising; and (2) dynamically estimating the SNR of both modalities based on denoising results to guide adaptive feature fusion, thereby injecting reliable cues into the diffusion process for high-fidelity visual reconstruction. Furthermore, we construct the REAL (Raw and Event Acquired in Low-light) dataset which provides 47,800 pixel-aligned low-light RAW images, events, and high-quality references under 0.001-0.8 lux illumination. Extensive experiments demonstrate the superiority of NEC-Diff under extreme darkness. The project are available at: https://github.com/jinghan-xu/NEC-Diff.
翻译:极低光照条件下动态场景的高质量成像极具挑战性。光子匮乏导致严重噪声与纹理缺失,造成图像显著退化。事件相机凭借高动态范围(120 dB)与运动高灵敏度特性,可通过提供保留细微纹理的关键线索成为传统相机的有力补充。然而,现有方法多聚焦于基于事件的纹理恢复,极少关注图像噪声或事件自身固有噪声,这最终阻碍了光子匮乏条件下的精确像素重建。本文提出NEC-Diff——一种基于扩散的事件-RAW混合成像框架,能够从严重噪声信号中提取可靠信息以重建精细场景结构。该框架基于两个关键洞察:(1)融合RAW图像的线性光响应特性与事件的亮度变化特性,构建物理驱动的鲁棒双模态去噪约束;(2)基于去噪结果动态估计双模态信噪比以引导自适应特征融合,从而将可靠线索注入扩散过程实现高保真视觉重建。此外,我们构建了REAL(极暗光下采集的RAW与事件)数据集,提供在0.001-0.8勒克斯照度下配准的47,800组低光照RAW图像、事件及高质量参考图像。大量实验证明了NEC-Diff在极暗条件下的优越性能。项目地址:https://github.com/jinghan-xu/NEC-Diff。