Modeling and synthesizing low-light raw noise is a fundamental problem for computational photography and image processing applications. Although most recent works have adopted physics-based models to synthesize noise, the signal-independent noise in low-light conditions is far more complicated and varies dramatically across camera sensors, which is beyond the description of these models. To address this issue, we introduce a new perspective to synthesize the signal-independent noise by a generative model. Specifically, we synthesize the signal-dependent and signal-independent noise in a physics- and learning-based manner, respectively. In this way, our method can be considered as a general model, that is, it can simultaneously learn different noise characteristics for different ISO levels and generalize to various sensors. Subsequently, we present an effective multi-scale discriminator termed Fourier transformer discriminator (FTD) to distinguish the noise distribution accurately. Additionally, we collect a new low-light raw denoising (LRD) dataset for training and benchmarking. Qualitative validation shows that the noise generated by our proposed noise model can be highly similar to the real noise in terms of distribution. Furthermore, extensive denoising experiments demonstrate that our method performs favorably against state-of-the-art methods on different sensors. The source code and dataset can be found at ~\url{https://github.com/fengzhang427/LRD}.
翻译:低光照原始噪声的建模与合成是计算摄影和图像处理应用中的基础问题。尽管近期多数研究采用基于物理的模型来合成噪声,但低光照条件下的信号无关噪声更为复杂,且不同相机制感器之间差异显著,超出了这些模型的描述范围。为解决这一问题,我们提出了一种新视角,通过生成模型合成信号无关噪声。具体而言,我们分别以基于物理和基于学习的方式合成信号相关噪声和信号无关噪声。由此,我们的方法可视为一种通用模型,能够同时学习不同ISO级别下的噪声特性,并泛化至多种传感器。随后,我们提出了一种有效的多尺度判别器——傅里叶变换判别器(FTD),以精确区分噪声分布。此外,我们收集了一个新的低光照原始去噪数据集(LRD),用于训练和基准测试。定性验证表明,我们提出的噪声模型生成的噪声在分布上与真实噪声高度相似。进一步的去噪实验表明,我们的方法在不同传感器上的表现优于现有最先进方法。源代码和数据集可在~\url{https://github.com/fengzhang427/LRD} 获取。