Adverse conditions typically suffer from stochastic hybrid weather degradations (e.g., rainy and hazy night), while existing image restoration algorithms envisage that weather degradations occur independently, thus may fail to handle real-world complicated scenarios. Besides, supervised training is not feasible due to the lack of a comprehensive paired dataset to characterize hybrid conditions. To this end, we have advanced the aforementioned limitations with two tactics: framework and data. First, we present a novel unified framework, dubbed RAHC, to Restore Arbitrary Hybrid adverse weather Conditions in one go. Specifically, our RAHC leverages a multi-head aggregation architecture to learn multiple degradation representation subspaces and then constrains the network to flexibly handle multiple hybrid adverse weather in a unified paradigm through a discrimination mechanism in the output space. Furthermore, we devise a reconstruction vectors aided scheme to provide auxiliary visual content cues for reconstruction, thus can comfortably cope with hybrid scenarios with insufficient remaining image constituents. Second, we construct a new dataset, termed HAC, for learning and benchmarking arbitrary Hybrid Adverse Conditions restoration. HAC contains 31 scenarios composed of an arbitrary combination of five common weather, with a total of ~316K adverse-weather/clean pairs. Extensive experiments yield superior results and establish new state-of-the-art results on both HAC and conventional datasets.
翻译:恶劣天气条件通常伴随随机的混合天气退化(例如雨雾交加的夜晚),而现有图像复原算法假定天气退化独立发生,因此可能无法处理现实中的复杂场景。此外,由于缺乏描述混合条件的全面配对数据集,监督训练并不可行。为此,我们通过框架和数据两个策略推进了上述局限的解决。首先,我们提出一种新颖的统一框架,命名为RAHC,用于一次性复原任意混合恶劣天气条件。具体而言,我们的RAHC利用多头聚合架构学习多个退化表示子空间,然后通过输出空间中的判别机制约束网络,以统一范式灵活处理多种混合恶劣天气。此外,我们设计了一种重构向量辅助方案,为重构提供辅助视觉内容线索,从而能够轻松应对剩余图像成分不足的混合场景。其次,我们构建了一个新数据集,称为HAC,用于学习和基准测试任意混合恶劣天气条件的复原。HAC包含31个场景,由五种常见天气的任意组合构成,总计约31.6万对恶劣天气/清洁图像。大量实验在HAC及传统数据集上均取得了优异结果,并确立了新的最先进性能。