The development of safe and reliable autonomous unmanned aerial vehicles relies on the ability of the system to recognise and adapt to changes in the local environment based on sensor inputs. State-of-the-art local tracking and trajectory planning are typically performed using camera sensor input to the flight control algorithm, but the extent to which environmental disturbances like rain affect the performance of these systems is largely unknown. In this paper, we first describe the development of an open dataset comprising ~335k images to examine these effects for seven different classes of precipitation conditions and show that a worst-case average tracking error of 1.5 m is possible for a state-of-the-art visual odometry system (VINS-Fusion). We then use the dataset to train a set of deep neural network models suited to mobile and constrained deployment scenarios to determine the extent to which it may be possible to efficiently and accurately classify these `rainy' conditions. The most lightweight of these models (MobileNetV3 small) can achieve an accuracy of 90% with a memory footprint of just 1.28 MB and a frame rate of 93 FPS, which is suitable for deployment in resource-constrained and latency-sensitive systems. We demonstrate a classification latency in the order of milliseconds using typical flight computer hardware. Accordingly, such a model can feed into the disturbance estimation component of an autonomous flight controller. In addition, data from unmanned aerial vehicles with the ability to accurately determine environmental conditions in real time may contribute to developing more granular timely localised weather forecasting.

翻译：安全可靠的自主无人机的开发依赖于系统根据传感器输入识别并适应局部环境变化的能力。最先进的局部跟踪与轨迹规划通常通过向飞行控制算法输入相机传感器数据来实现，但诸如降雨等环境干扰对这些系统性能的影响程度在很大程度上尚不明确。本文首先描述了一个包含约33.5万张图像的开源数据集的构建过程，该数据集用于研究七类不同降水条件的影响，并表明对于最先进的视觉里程计系统（VINS-Fusion），在最坏情况下可能产生平均1.5米的跟踪误差。随后，我们利用该数据集训练了一系列适用于移动及受限部署场景的深度神经网络模型，以评估高效准确分类这些"降雨"条件的可行性。其中最轻量级的模型（MobileNetV3 small）可实现90%的准确率，内存占用仅为1.28 MB，帧率达到93 FPS，适合部署在资源受限且对延迟敏感的系统。我们在典型飞行计算机硬件上展示了毫秒级的分类延迟。因此，此类模型可集成至自主飞行控制器的扰动估计模块。此外，具备实时准确判断环境条件能力的无人机所采集的数据，可能有助于开发更精细的即时局部天气预报。