Vision in adverse weather conditions, whether it be snow, rain, or fog is challenging. In these scenarios, scattering and attenuation severly degrades image quality. Handling such inclement weather conditions, however, is essential to operate autonomous vehicles, drones and robotic applications where human performance is impeded the most. A large body of work explores removing weather-induced image degradations with dehazing methods. Most methods rely on single images as input and struggle to generalize from synthetic fully-supervised training approaches or to generate high fidelity results from unpaired real-world datasets. With data as bottleneck and most of today's training data relying on good weather conditions with inclement weather as outlier, we rely on an inverse rendering approach to reconstruct the scene content. We introduce ScatterNeRF, a neural rendering method which adequately renders foggy scenes and decomposes the fog-free background from the participating media-exploiting the multiple views from a short automotive sequence without the need for a large training data corpus. Instead, the rendering approach is optimized on the multi-view scene itself, which can be typically captured by an autonomous vehicle, robot or drone during operation. Specifically, we propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.

翻译：恶劣天气条件（无论是雪、雨还是雾）下的视觉感知极具挑战性。在这些场景中，散射与衰减会严重降低图像质量。然而，处理此类恶劣天气对于运行自动驾驶汽车、无人机及机器人应用至关重要——而这些恰恰是人类性能最受阻碍的领域。大量研究通过去雾方法探索消除天气造成的图像退化问题。多数方法依赖单张图像输入，且难以从全监督合成训练方法中实现泛化，或无法从未配对真实数据集中生成高保真结果。考虑到数据是瓶颈，且当前大多数训练数据依赖良好天气条件（恶劣天气为特例），我们采用逆向渲染方法重建场景内容。本文提出ScatterNeRF——一种神经渲染方法，能够恰当渲染雾天场景，并利用短程汽车序列中的多视角将无雾背景与参与介质解耦，无需大规模训练数据集。该渲染方法转而针对多视角场景自身进行优化，而这些视角通常可在自动驾驶汽车、机器人或无人机运行期间采集。具体而言，我们为散射体积与场景对象提出解耦表示，并通过物理启发的损失函数学习场景重建。通过采集多视角野外数据及大规模雾室中的受控捕获数据，我们对方法进行了验证。