Existing Video Restoration (VR) methods always necessitate the individual deployment of models for each adverse weather to remove diverse adverse weather degradations, lacking the capability for adaptive processing of degradations. Such limitation amplifies the complexity and deployment costs in practical applications. To overcome this deficiency, in this paper, we propose a Cross-consistent Deep Unfolding Network (CDUN) for All-In-One VR, which enables the employment of a single model to remove diverse degradations for the first time. Specifically, the proposed CDUN accomplishes a novel iterative optimization framework, capable of restoring frames corrupted by corresponding degradations according to the degradation features given in advance. To empower the framework for eliminating diverse degradations, we devise a Sequence-wise Adaptive Degradation Estimator (SADE) to estimate degradation features for the input corrupted video. By orchestrating these two cascading procedures, CDUN achieves adaptive processing for diverse degradation. In addition, we introduce a window-based inter-frame fusion strategy to utilize information from more adjacent frames. This strategy involves the progressive stacking of temporal windows in multiple iterations, effectively enlarging the temporal receptive field and enabling each frame's restoration to leverage information from distant frames. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance in All-In-One VR.
翻译:现有视频恢复方法通常需要针对每种恶劣天气单独部署模型以去除各种不良天气退化,缺乏自适应处理退化的能力。这一局限性在实际应用中增加了复杂性和部署成本。为克服此不足,本文提出了一种跨一致性深度展开网络(CDUN)用于全合一视频恢复,首次实现了单一模型去除多种退化。具体而言,所提出的CDUN构建了一个新颖的迭代优化框架,能够根据预先给定的退化特征恢复被相应退化破坏的视频帧。为使该框架能够消除多种退化,我们设计了一种序列自适应退化估计器(SADE),用于估计输入损坏视频的退化特征。通过协调这两个级联过程,CDUN实现了对多种退化的自适应处理。此外,我们引入了一种基于窗口的帧间融合策略,以利用更多相邻帧的信息。该策略通过在多次迭代中逐步堆叠时间窗口,有效扩大了时间感受野,使每帧的恢复能够利用远距离帧的信息。大量实验表明,所提出的方法在全合一视频恢复中达到了最优性能。