Modern consumer cameras usually employ the rolling shutter (RS) mechanism, where images are captured by scanning scenes row-by-row, yielding RS distortions for dynamic scenes. To correct RS distortions, existing methods adopt a fully supervised learning manner, where high framerate global shutter (GS) images should be collected as ground-truth supervision. In this paper, we propose a Self-supervised learning framework for Dual reversed RS distortions Correction (SelfDRSC), where a DRSC network can be learned to generate a high framerate GS video only based on dual RS images with reversed distortions. In particular, a bidirectional distortion warping module is proposed for reconstructing dual reversed RS images, and then a self-supervised loss can be deployed to train DRSC network by enhancing the cycle consistency between input and reconstructed dual reversed RS images. Besides start and end RS scanning time, GS images at arbitrary intermediate scanning time can also be supervised in SelfDRSC, thus enabling the learned DRSC network to generate a high framerate GS video. Moreover, a simple yet effective self-distillation strategy is introduced in self-supervised loss for mitigating boundary artifacts in generated GS images. On synthetic dataset, SelfDRSC achieves better or comparable quantitative metrics in comparison to state-of-the-art methods trained in the full supervision manner. On real-world RS cases, our SelfDRSC can produce high framerate GS videos with finer correction textures and better temporary consistency. The source code and trained models are made publicly available at https://github.com/shangwei5/SelfDRSC. We also provide an implementation in HUAWEI Mindspore at https://github.com/Hunter-Will/SelfDRSC-mindspore.

翻译：现代消费相机通常采用卷帘快门机制，通过逐行扫描场景捕获图像，导致动态场景产生卷帘快门畸变。现有畸变校正方法采用全监督学习方式，需采集高帧率全局快门图像作为真值监督。本文提出一种针对双反向卷帘快门畸变校正的自监督学习框架（SelfDRSC），仅基于具有反向畸变的双卷帘快门图像即可学习生成高帧率全局快门视频。具体而言，我们构建了双向畸变扭曲模块用于重建双反向卷帘快门图像，进而通过增强输入与重建双反向卷帘快门图像之间的循环一致性，设计自监督损失函数训练DRSC网络。除起始和结束卷帘快门扫描时刻外，SelfDRSC还能对任意中间扫描时刻的全局快门图像进行监督，使训练后的DRSC网络可生成高帧率全局快门视频。此外，我们在自监督损失中引入简洁有效的自蒸馏策略，以抑制生成全局快门图像的边界伪影。在合成数据集上，SelfDRSC相比全监督训练的最优方法取得了相当或更优的量化指标。在真实世界卷帘快门案例中，SelfDRSC可生成具有更精细校正纹理与更优时间一致性的高帧率全局快门视频。源代码与训练模型已开源至https://github.com/shangwei5/SelfDRSC，同时在华为MindSpore框架下的实现见https://github.com/Hunter-Will/SelfDRSC-mindspore。