Weather recognition is an essential support for many practical life applications, including traffic safety, environment, and meteorology. However, many existing related works cannot comprehensively describe weather conditions due to their complex co-occurrence dependencies. This paper proposes a novel multi-label weather recognition model considering these dependencies. The proposed model called MASK-Convolutional Neural Network-Transformer (MASK-CT) is based on the Transformer, the convolutional process, and the MASK mechanism. The model employs multiple convolutional layers to extract features from weather images and a Transformer encoder to calculate the probability of each weather condition based on the extracted features. To improve the generalization ability of MASK-CT, a MASK mechanism is used during the training phase. The effect of the MASK mechanism is explored and discussed. The Mask mechanism randomly withholds some information from one-pair training instances (one image and its corresponding label). There are two types of MASK methods. Specifically, MASK-I is designed and deployed on the image before feeding it into the weather feature extractor and MASK-II is applied to the image label. The Transformer encoder is then utilized on the randomly masked image features and labels. The experimental results from various real-world weather recognition datasets demonstrate that the proposed MASK-CT model outperforms state-of-the-art methods. Furthermore, the high-speed dynamic real-time weather recognition capability of the MASK-CT is evaluated.

翻译：摘要：天气识别是诸多实际生活应用（包括交通安全、环境及气象领域）的关键支撑。然而，现有相关工作由于难以刻画天气现象间复杂的共现依赖关系，往往无法全面描述天气状态。本文提出一种考虑此类依赖关系的新型多标签天气识别模型。该模型名为掩码卷积神经网络-Transformer（MASK-CT），基于Transformer架构、卷积处理过程与MASK机制构建。模型采用多层卷积层提取天气图像特征，并通过Transformer编码器基于所提取特征计算每种天气状态的概率。为提升MASK-CT的泛化能力，训练阶段引入MASK机制，并对其效果进行了探索与讨论。该机制随机隐藏配对训练实例（单张图像及其对应标签）中的部分信息。MASK方法包含两种类型：具体而言，MASK-I在图像输入天气特征提取器前对其施加掩码操作，MASK-II则应用于图像标签。随后利用Transformer编码器处理随机掩码后的图像特征与标签。基于多种真实世界天气识别数据集的实验结果表明，所提出的MASK-CT模型性能优于现有最优方法。此外，本文还评估了MASK-CT的高速动态实时天气识别能力。