Semantic segmentation is a challenging task in the absence of densely labelled data. Only relying on class activation maps (CAM) with image-level labels provides deficient segmentation supervision. Prior works thus consider pre-trained models to produce coarse saliency maps to guide the generation of pseudo segmentation labels. However, the commonly used off-line heuristic generation process cannot fully exploit the benefits of these coarse saliency maps. Motivated by the significant inter-task correlation, we propose a novel weakly supervised multi-task framework termed as AuxSegNet, to leverage saliency detection and multi-label image classification as auxiliary tasks to improve the primary task of semantic segmentation using only image-level ground-truth labels. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks. The mutual boost between pseudo label updating and cross-task affinity learning enables iterative improvements on segmentation performance. Extensive experiments demonstrate the effectiveness of the proposed auxiliary learning network structure and the cross-task affinity learning method. The proposed approach achieves state-of-the-art weakly supervised segmentation performance on the challenging PASCAL VOC 2012 and MS COCO benchmarks.

翻译：在缺少密集标签数据的情况下,语义分解是一项艰巨的任务。 只有在依赖带有图像等级标签的类别激活地图( CAM) 的情况下, 才会提供不完善的分解监督。 先前的工作将先考虑经过培训的模型, 以生成粗显性图, 指导伪分解标签的生成。 然而, 通常使用的离线超光层生成过程无法充分利用这些粗显性图的惠益。 受重要的任务间关联的驱动, 我们提议建立一个名为 AuxSegNet 的新颖的、 监管不力的多任务框架, 以利用显著的检测和多标签图像分类作为辅助任务, 仅使用图像级别地义标签标签改进语义分解的主要任务。 受类似结构结构的语义学启发, 我们还提议从突出分层图和分层图中学习一个跨星系全球等级相近性图。 所学的跨任务跨任务组合的跨任务缩略缩略图可以用来改进突出的预测, 传播CAM 地图, 为这两项任务提供更好的假称标签。 假称标签更新和跨级标签之间的相互推介点 更新和跨结构的跨结构的跨结构 学习拟议中 学习的跨结构 学习系统 学习系统 学习系统 学习系统 学习系统 学习 学习 学习 度 度 度 度 度