This paper explores the properties of the plain Vision Transformer (ViT) for Weakly-supervised Semantic Segmentation (WSSS). The class activation map (CAM) is of critical importance for understanding a classification network and launching WSSS. We observe that different attention heads of ViT focus on different image areas. Thus a novel weight-based method is proposed to end-to-end estimate the importance of attention heads, while the self-attention maps are adaptively fused for high-quality CAM results that tend to have more complete objects. Besides, we propose a ViT-based gradient clipping decoder for online retraining with the CAM results to complete the WSSS task. We name this plain Transformer-based Weakly-supervised learning framework WeakTr. It achieves the state-of-the-art WSSS performance on standard benchmarks, i.e., 78.4% mIoU on the val set of PASCAL VOC 2012 and 50.3% mIoU on the val set of COCO 2014. Code is available at https://github.com/hustvl/WeakTr.

翻译：本文探讨了普通视觉Transformer（ViT）在弱监督语义分割中的特性。类激活映射对于理解分类网络和启动弱监督语义分割至关重要。我们观察到ViT的不同注意力头关注不同的图像区域。因此，提出了一种新颖的基于权重的端到端方法评估注意力头的重要性，同时自适应融合自注意力映射以生成倾向于包含更完整物体的高质量CAM结果。此外，我们提出了一种基于ViT的梯度裁剪解码器，用于利用CAM结果进行在线重训练以完成弱监督语义分割任务。我们将这种基于普通Transformer的弱监督学习框架命名为WeakTr。它在标准基准测试中取得了最先进的弱监督语义分割性能，即在PASCAL VOC 2012验证集上达到78.4%的mIoU，在COCO 2014验证集上达到50.3%的mIoU。代码可在https://github.com/hustvl/WeakTr获取。