Optical flow has achieved great success under clean scenes, but suffers from restricted performance under foggy scenes. To bridge the clean-to-foggy domain gap, the existing methods typically adopt the domain adaptation to transfer the motion knowledge from clean to synthetic foggy domain. However, these methods unexpectedly neglect the synthetic-to-real domain gap, and thus are erroneous when applied to real-world scenes. To handle the practical optical flow under real foggy scenes, in this work, we propose a novel unsupervised cumulative domain adaptation optical flow (UCDA-Flow) framework: depth-association motion adaptation and correlation-alignment motion adaptation. Specifically, we discover that depth is a key ingredient to influence the optical flow: the deeper depth, the inferior optical flow, which motivates us to design a depth-association motion adaptation module to bridge the clean-to-foggy domain gap. Moreover, we figure out that the cost volume correlation shares similar distribution of the synthetic and real foggy images, which enlightens us to devise a correlation-alignment motion adaptation module to distill motion knowledge of the synthetic foggy domain to the real foggy domain. Note that synthetic fog is designed as the intermediate domain. Under this unified framework, the proposed cumulative adaptation progressively transfers knowledge from clean scenes to real foggy scenes. Extensive experiments have been performed to verify the superiority of the proposed method.
翻译:光流在清晰场景下已取得巨大成功,但在雾天场景下性能受限。为弥合清晰到合成雾天的域差异,现有方法通常采用域适应策略将运动知识从清晰域迁移至合成雾天域。然而,这些方法意外忽视了合成域到真实域之间的鸿沟,因此在应用于真实场景时会产生误差。为解决真实雾天场景下的实用光流估计问题,本文提出一种新颖的无监督累积域适应光流框架(UCDA-Flow),包含深度关联运动适应模块与相关性对齐运动适应模块。具体而言,我们发现深度是影响光流的关键因素:深度越大,光流质量越差,这启发我们设计深度关联运动适应模块以弥合清晰域与合成雾天域的差异。进一步,我们发现代价体相关性在合成雾天图像与真实雾天图像中具有相似分布,这促使我们设计相关性对齐运动适应模块,将合成雾天域的运动知识蒸馏至真实雾天域。值得注意的是,合成雾被设计为中间域。在该统一框架下,所提出的累积适应机制逐步将知识从清晰场景迁移至真实雾天场景。大量实验验证了所提方法的优越性。