The top-down and bottom-up methods are two mainstreams of referring segmentation, while both methods have their own intrinsic weaknesses. Top-down methods are chiefly disturbed by Polar Negative (PN) errors owing to the lack of fine-grained cross-modal alignment. Bottom-up methods are mainly perturbed by Inferior Positive (IP) errors due to the lack of prior object information. Nevertheless, we discover that two types of methods are highly complementary for restraining respective weaknesses but the direct average combination leads to harmful interference. In this context, we build Win-win Cooperation (WiCo) to exploit complementary nature of two types of methods on both interaction and integration aspects for achieving a win-win improvement. For the interaction aspect, Complementary Feature Interaction (CFI) provides fine-grained information to top-down branch and introduces prior object information to bottom-up branch for complementary feature enhancement. For the integration aspect, Gaussian Scoring Integration (GSI) models the gaussian performance distributions of two branches and weightedly integrates results by sampling confident scores from the distributions. With our WiCo, several prominent top-down and bottom-up combinations achieve remarkable improvements on three common datasets with reasonable extra costs, which justifies effectiveness and generality of our method.
翻译:自上而下方法和自底向上方法是指代分割的两大主流范式,但两者均存在固有缺陷。自上而下方法因缺乏细粒度跨模态对齐而主要受极性负(PN)误差干扰;自底向上方法因缺少先验目标信息而主要受低质正(IP)误差扰动。然而,我们发现两类方法在抑制各自缺陷方面具有高度互补性,但直接平均组合会导致有害干扰。在此背景下,我们构建双赢协作(WiCo)机制,从交互和融合两个维度挖掘两类方法的互补特性以实现共赢提升。在交互维度,互补特征交互(CFI)为自上而下分支提供细粒度信息,同时为自底向上分支引入先验目标信息,实现互补特征增强;在融合维度,高斯评分融合(GSI)对两个分支的高斯性能分布进行建模,并通过从分布中采样置信度分数进行加权融合。采用WiCo方法后,多种经典自上而下与自底向上组合在三个公开数据集上以合理额外成本取得显著性能提升,验证了本方法的有效性与通用性。