Capturing screen contents by smartphone cameras has become a common way for information sharing. However, these images and videos are often degraded by moir\'e patterns, which are caused by frequency aliasing between the camera filter array and digital display grids. We observe that the moir\'e patterns in raw domain is simpler than those in sRGB domain, and the moir\'e patterns in raw color channels have different properties. Therefore, we propose an image and video demoir\'eing network tailored for raw inputs. We introduce a color-separated feature branch, and it is fused with the traditional feature-mixed branch via channel and spatial modulations. Specifically, the channel modulation utilizes modulated color-separated features to enhance the color-mixed features. The spatial modulation utilizes the feature with large receptive field to modulate the feature with small receptive field. In addition, we build the first well-aligned raw video demoir\'eing (RawVDemoir\'e) dataset and propose an efficient temporal alignment method by inserting alternating patterns. Experiments demonstrate that our method achieves state-of-the-art performance for both image and video demori\'eing. We have released the code and dataset in https://github.com/tju-chengyijia/VD_raw.
翻译:通过智能手机相机拍摄屏幕内容已成为信息共享的常见方式。然而,这些图像和视频常因相机滤波阵列与数字显示网格之间的频率混叠而产生摩尔纹退化。我们观察到原始域中的摩尔纹模式比sRGB域更简单,且原始颜色通道中的摩尔纹具有不同特性。为此,我们提出了一种专为原始输入设计的图像与视频去摩尔纹网络。该网络引入颜色分离特征分支,并通过通道调制与空间调制将其与传统的特征混合分支融合。具体而言,通道调制利用经过调制的颜色分离特征增强颜色混合特征,空间调制则利用大感受野特征调节小感受野特征。此外,我们构建了首个严格对齐的原始视频去摩尔纹(RawVDemoiré)数据集,并提出通过插入交替模式实现高效时序对齐的方法。实验证明,本方法在图像与视频去摩尔纹任务中均达到最优性能。我们已在 https://github.com/tju-chengyijia/VD_raw 开源代码与数据集。