Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distribution, while learning-based noise modeling impractically depends on paired real data. In this paper, we propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency. Based on this strategy, we introduce an efficient physics-informed noise neural proxy (PNNP) to approximate the real-world sensor noise model. Specifically, we integrate physical priors into neural proxies and introduce three efficient techniques: physics-guided noise decoupling (PND), physics-aware proxy model (PPM), and differentiable distribution loss (DDL). PND decouples the dark frame into different components and handles different levels of noise flexibly, which reduces the complexity of noise modeling. PPM incorporates physical priors to constrain the synthetic noise, which promotes the accuracy of noise modeling. DDL provides explicit and reliable supervision for noise distribution, which promotes the precision of noise modeling. PNNP exhibits powerful potential in characterizing the real noise distribution. Extensive experiments on public datasets demonstrate superior performance in practical low-light raw image denoising. The source code will be publicly available at the project homepage.
翻译:近年来,训练低光照原始图像去噪方法的主流实践已转向使用合成数据。噪声建模——其核心在于刻画真实世界传感器的噪声分布——深刻影响着合成数据的有效性与实用性。当前,基于物理的噪声建模难以完整刻画真实噪声分布,而基于学习的噪声建模则不切实际地依赖于配对的真实数据。本文提出一种新颖策略:从暗帧而非配对真实数据中学习噪声模型,以突破数据依赖性。基于此策略,我们引入一种高效的物理信息噪声神经代理(PNNP)来逼近真实世界传感器噪声模型。具体而言,我们将物理先验整合到神经代理中,并引入三项高效技术:物理引导噪声解耦(PND)、物理感知代理模型(PPM)以及可微分分布损失(DDL)。PND将暗帧解耦为不同分量并灵活处理不同层级的噪声,从而降低了噪声建模的复杂度。PPM通过融入物理先验来约束合成噪声,提升了噪声建模的准确性。DDL为噪声分布提供显式且可靠的监督,促进了噪声建模的精确性。PNNP在刻画真实噪声分布方面展现出强大潜力。在公开数据集上的大量实验表明,该方法在实际低光照原始图像去噪任务中具有卓越性能。源代码将在项目主页公开。