Current adversarial attacks on motion estimation, or optical flow, optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, adverse weather conditions constitute a much more realistic threat scenario. Hence, in this work, we present a novel attack on motion estimation that exploits adversarially optimized particles to mimic weather effects like snowflakes, rain streaks or fog clouds. At the core of our attack framework is a differentiable particle rendering system that integrates particles (i) consistently over multiple time steps (ii) into the 3D space (iii) with a photo-realistic appearance. Through optimization, we obtain adversarial weather that significantly impacts the motion estimation. Surprisingly, methods that previously showed good robustness towards small per-pixel perturbations are particularly vulnerable to adversarial weather. At the same time, augmenting the training with non-optimized weather increases a method's robustness towards weather effects and improves generalizability at almost no additional cost. Our code will be available at https://github.com/cv-stuttgart/DistractingDownpour.

翻译：当前针对运动估计（即光流）的对抗性攻击通常优化微小的逐像素扰动，这类扰动在真实世界中难以出现。相比之下，恶劣天气条件构成了更具现实意义的威胁场景。因此，本研究提出了一种新颖的运动估计攻击方法，该方法利用对抗性优化粒子模拟雪片、雨丝或雾团等天气效应。我们攻击框架的核心是一个可微粒子渲染系统，它能够将粒子（i）跨多个时间步长连贯地集成到（ii）三维空间中，并具备（iii）照片级逼真的外观。通过优化，我们获得了显著影响运动估计的对抗性天气。令人惊讶的是，此前展示出对逐像素扰动具有良好鲁棒性的方法，在面对对抗性天气时反而显得尤为脆弱。同时，在训练中引入未优化的天气增强数据，能够以几乎零额外成本提升方法对天气效应的鲁棒性，并改善其泛化能力。我们的代码将在https://github.com/cv-stuttgart/DistractingDownpour 公开。