Cross-domain few-shot segmentation (CD-FSS) aims to achieve semantic segmentation in previously unseen domains with a limited number of annotated samples. Although existing CD-FSS models focus on cross-domain feature transformation, relying exclusively on inter-domain knowledge transfer may lead to the loss of critical intra-domain information. To this end, we propose a novel residual transformation network (RestNet) that facilitates knowledge transfer while retaining the intra-domain support-query feature information. Specifically, we propose a Semantic Enhanced Anchor Transform (SEAT) module that maps features to a stable domain-agnostic space using advanced semantics. Additionally, an Intra-domain Residual Enhancement (IRE) module is designed to maintain the intra-domain representation of the original discriminant space in the new space. We also propose a mask prediction strategy based on prototype fusion to help the model gradually learn how to segment. Our RestNet can transfer cross-domain knowledge from both inter-domain and intra-domain without requiring additional fine-tuning. Extensive experiments on ISIC, Chest X-ray, and FSS-1000 show that our RestNet achieves state-of-the-art performance. Our code will be available soon.
翻译:摘要:跨域小样本分割旨在通过有限标注样本实现对未见领域的语义分割。现有跨域小样本分割模型虽聚焦于跨域特征变换,但仅依赖域间知识迁移可能导致关键域内信息的丢失。为此,我们提出一种新型残差变换网络,在促进知识迁移的同时保留域内支持-查询特征信息。具体而言,我们设计了语义增强锚点变换模块,利用高级语义将特征映射至稳定的域无关空间;同时提出域内残差增强模块,在新空间中维持原始判别空间的域内表征。此外,我们提出基于原型融合的掩码预测策略,帮助模型逐步学习分割能力。我们的RestNet无需额外微调即可从域间和域内两个维度实现跨域知识迁移。在ISIC、Chest X-ray及FSS-1000数据集上的大量实验表明,RestNet达到了最优性能。相关代码即将开源。