Clouds are a common phenomenon that distorts optical satellite imagery, which poses a challenge for remote sensing. However, in the literature cloudless analysis is often performed where cloudy images are excluded from machine learning datasets and methods. Such an approach cannot be applied to time sensitive applications, e.g., during natural disasters. A possible solution is to apply cloud removal as a preprocessing step to ensure that cloudfree solutions are not failing under such conditions. But cloud removal methods are still actively researched and suffer from drawbacks, such as generated visual artifacts. Therefore, it is desirable to develop cloud robust methods that are less affected by cloudy weather. Cloud robust methods can be achieved by combining optical data with radar, a modality unaffected by clouds. While many datasets for machine learning combine optical and radar data, most researchers exclude cloudy images. We identify this exclusion from machine learning training and evaluation as a limitation that reduces applicability to cloudy scenarios. To investigate this, we assembled a dataset, named CloudyBigEarthNet (CBEN), of paired optical and radar images with cloud occlusion for training and evaluation. Using average precision (AP) as the evaluation metric, we show that state-of-the-art methods trained on combined clear-sky optical and radar imagery suffer performance drops of 23-33 percentage points when evaluated on cloudy images. We then adapt these methods to cloudy optical data during training, achieving relative improvement of 17.2-28.7 percentage points on cloudy test cases compared with the original approaches. Code and dataset are publicly available at: https://github.com/mstricker13/CBEN

翻译：云层是扭曲光学卫星图像的常见现象，这对遥感应用构成了挑战。然而，现有文献中常进行无云分析，即将含云图像排除在机器学习数据集和方法之外。此类方法无法应用于时间敏感的应用场景，例如在自然灾害期间。一种可能的解决方案是将云去除作为预处理步骤，以确保无云解决方案在此类条件下不会失效。但云去除方法仍处于积极研究阶段，且存在诸如生成视觉伪影等缺陷。因此，开发受云层天气影响较小的云稳健方法具有重要价值。通过将光学数据与不受云层影响的雷达模态相结合，可以实现云稳健方法。尽管许多机器学习数据集结合了光学和雷达数据，但大多数研究者仍排除含云图像。我们认为，在机器学习训练和评估中排除含云图像是一种局限性，会降低方法在含云场景下的适用性。为研究此问题，我们构建了一个名为CloudyBigEarthNet（CBEN）的数据集，其中包含用于训练和评估的成对光学与雷达含云遮挡图像。以平均精度（AP）作为评估指标，我们表明，在清晰天空光学与雷达图像上训练的最先进方法，在含云图像上评估时性能下降达23-33个百分点。随后，我们在训练过程中将这些方法适配至含云光学数据，相比原始方法，在含云测试案例上实现了17.2-28.7个百分点的相对性能提升。代码和数据集已公开于：https://github.com/mstricker13/CBEN