Current adversarial attacks on motion estimation, or optical flow, optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, adverse weather conditions constitute a much more realistic threat scenario. Hence, in this work, we present a novel attack on motion estimation that exploits adversarially optimized particles to mimic weather effects like snowflakes, rain streaks or fog clouds. At the core of our attack framework is a differentiable particle rendering system that integrates particles (i) consistently over multiple time steps (ii) into the 3D space (iii) with a photo-realistic appearance. Through optimization, we obtain adversarial weather that significantly impacts the motion estimation. Surprisingly, methods that previously showed good robustness towards small per-pixel perturbations are particularly vulnerable to adversarial weather. At the same time, augmenting the training with non-optimized weather increases a method's robustness towards weather effects and improves generalizability at almost no additional cost.

翻译：当前针对运动估计（或光流）的对抗性攻击通常优化微小的逐像素扰动，这类扰动在现实世界中几乎不会出现。相比之下，恶劣天气条件构成更具现实意义的威胁场景。因此，本文提出一种新颖的运动估计攻击方法，利用对抗优化的粒子模拟雪花、雨线或雾团等天气效应。我们攻击框架的核心是可微粒子渲染系统，该系统能将粒子（i）在多个时间步长上保持一致性，（ii）集成到三维空间中，（iii）并具有照片级真实感外观。通过优化，我们获得了能显著影响运动估计的对抗性天气。令人惊讶的是，此前在微小逐像素扰动下展现出良好鲁棒性的方法，反而特别容易受到对抗性天气的攻击。同时，使用非优化的天气进行训练增强，能以几乎为零的额外成本提升方法对天气效应的鲁棒性并改善泛化能力。