In recent years, weakly supervised semantic segmentation using image-level labels as supervision has received significant attention in the field of computer vision. Most existing methods have addressed the challenges arising from the lack of spatial information in these labels by focusing on facilitating supervised learning through the generation of pseudo-labels from class activation maps (CAMs). Due to the localized pattern detection of CNNs, CAMs often emphasize only the most discriminative parts of an object, making it challenging to accurately distinguish foreground objects from each other and the background. Recent studies have shown that Vision Transformer (ViT) features, due to their global view, are more effective in capturing the scene layout than CNNs. However, the use of hierarchical ViTs has not been extensively explored in this field. This work explores the use of Swin Transformer by proposing "SWTformer" to enhance the accuracy of the initial seed CAMs by bringing local and global views together. SWTformer-V1 generates class probabilities and CAMs using only the patch tokens as features. SWTformer-V2 incorporates a multi-scale feature fusion mechanism to extract additional information and utilizes a background-aware mechanism to generate more accurate localization maps with improved cross-object discrimination. Based on experiments on the PascalVOC 2012 dataset, SWTformer-V1 achieves a 0.98% mAP higher localization accuracy, outperforming state-of-the-art models. It also yields comparable performance by 0.82% mIoU on average higher than other methods in generating initial localization maps, depending only on the classification network. SWTformer-V2 further improves the accuracy of the generated seed CAMs by 5.32% mIoU, further proving the effectiveness of the local-to-global view provided by the Swin transformer. Code available at: https://github.com/RozhanAhmadi/SWTformer

翻译：近年来，以图像级标签为监督信息的弱监督语义分割在计算机视觉领域受到广泛关注。现有方法大多通过利用类激活图（CAM）生成伪标签来促进监督学习，以解决标签缺乏空间信息带来的挑战。由于卷积神经网络（CNN）的局部模式检测特性，CAM往往只突出物体的最具区分性部分，导致难以准确区分前景物体之间及其与背景的关系。近期研究表明，视觉Transformer（ViT）特征因其全局视野，在捕捉场景布局方面比CNN更有效。然而，层次化ViT在该领域尚未被充分探索。本研究通过提出"SWTformer"，利用Swin Transformer将局部与全局视图结合，以提升初始种子CAM的准确性。SWTformer-V1仅使用图像块令牌作为特征生成类别概率和CAM。SWTformer-V2引入多尺度特征融合机制提取额外信息，并利用背景感知机制生成更具交叉物体区分能力的精准定位图。基于PascalVOC 2012数据集的实验表明，SWTformer-V1的定位精度较现有最优模型提升了0.98%的平均精度（mAP），同时在仅依赖分类网络生成初始定位图时，平均交并比（mIoU）较其他方法高出0.82%，性能相当。SWTformer-V2进一步将生成种子CAM的精度提升5.32%的mIoU，充分验证了Swin Transformer提供的局部到全局视角的有效性。代码已开源：https://github.com/RozhanAhmadi/SWTformer