Many videos contain flickering artifacts. Common causes of flicker include video processing algorithms, video generation algorithms, and capturing videos under specific situations. Prior work usually requires specific guidance such as the flickering frequency, manual annotations, or extra consistent videos to remove the flicker. In this work, we propose a general flicker removal framework that only receives a single flickering video as input without additional guidance. Since it is blind to a specific flickering type or guidance, we name this "blind deflickering." The core of our approach is utilizing the neural atlas in cooperation with a neural filtering strategy. The neural atlas is a unified representation for all frames in a video that provides temporal consistency guidance but is flawed in many cases. To this end, a neural network is trained to mimic a filter to learn the consistent features (e.g., color, brightness) and avoid introducing the artifacts in the atlas. To validate our method, we construct a dataset that contains diverse real-world flickering videos. Extensive experiments show that our method achieves satisfying deflickering performance and even outperforms baselines that use extra guidance on a public benchmark.
翻译:许多视频中存在闪烁伪影。闪烁的常见原因包括视频处理算法、视频生成算法以及特定场景下的视频采集。以往工作通常需要特定指导信息,如闪烁频率、人工标注或额外的一致视频来消除闪烁。本文提出一种通用闪烁去除框架,仅以单个闪烁视频作为输入,无需额外指导。由于该方法对特定闪烁类型或引导信息具有盲适应性,我们称之为"盲去闪"。该方法的核心是利用神经图谱协同神经滤波策略。神经图谱是视频所有帧的统一表征,可提供时序一致性指导,但在许多情况下存在缺陷。为此,我们训练神经网络模拟滤波器以学习一致性特征(如颜色、亮度),同时避免引入图谱中的伪影。为验证方法有效性,我们构建了包含多样化真实场景闪烁视频的数据集。大量实验表明,本方法实现了令人满意的去闪效果,甚至在公开基准测试中超越了使用额外指导信息的基线方法。