One of the main tasks of an autonomous agent in a vehicle is to correctly perceive its environment. Much of the data that needs to be processed is collected by optical sensors such as cameras. Unfortunately, the data collected in this way can be affected by a variety of factors, including environmental influences such as inclement weather conditions (e.g., rain). Such noisy data can cause autonomous agents to take wrong decisions with potentially fatal outcomes. This paper addresses the rain image challenge by two steps: First, rain is artificially added to a set of clear-weather condition images using a Generative Adversarial Network (GAN). This yields good/bad weather image pairs for training de-raining models. This artificial generation of rain images is sufficiently realistic as in 7 out of 10 cases, human test subjects believed the generated rain images to be real. In a second step, this paired good/bad weather image data is used to train two rain denoising models, one based primarily on a Convolutional Neural Network (CNN) and the other using a Vision Transformer. This rain de-noising step showed limited performance as the quality gain was only about 15%. This lack of performance on realistic rain images as used in our study is likely due to current rain de-noising models being developed for simplistic rain overlay data. Our study shows that there is ample space for improvement of de-raining models in autonomous driving.

翻译：自动驾驶车辆的核心任务之一是准确感知周围环境。需要处理的大量数据由摄像头等光学传感器采集。然而，这类数据可能受到多种因素影响，包括恶劣天气条件（如降雨）等环境干扰。此类噪声数据可能导致自动驾驶代理做出错误决策，甚至引发致命后果。本文通过两个步骤解决雨图问题：首先，利用生成对抗网络（GAN）在晴朗天气图像集上人工添加雨滴，生成用于训练去雨模型的配对天气图像。这种人工生成的雨图具有足够真实性——在10个案例中有7例，人类受试者认为生成的雨图为真实场景。其次，使用配对的晴/雨天气图像数据训练两种去噪模型：一种基于卷积神经网络（CNN），另一种基于视觉Transformer（Vision Transformer）。去雨步骤的性能提升有限，质量增益仅约15%。该性能不足的原因可能在于当前去雨模型均针对简化降雨覆盖数据开发。本研究表明，自动驾驶领域的去雨模型仍有广阔的改进空间。