In this paper, we address the problem of sim-to-real transfer for object segmentation when there is no access to real examples of an object of interest during training, i.e. zero-shot sim-to-real transfer for segmentation. We focus on the application of shipwreck segmentation in side scan sonar imagery. Our novel segmentation network, STARS, addresses this challenge by fusing a predicted deformation field and anomaly volume, allowing it to generalize better to real sonar images and achieve more effective zero-shot sim-to-real transfer for image segmentation. We evaluate the sim-to-real transfer capabilities of our method on a real, expert-labeled side scan sonar dataset of shipwrecks collected from field work surveys with an autonomous underwater vehicle (AUV). STARS is trained entirely in simulation and performs zero-shot shipwreck segmentation with no additional fine-tuning on real data. Our method provides a significant 20% increase in segmentation performance for the targeted shipwreck class compared to the best baseline.

翻译：本文针对目标分割中仿真到现实迁移问题展开研究，其核心挑战在于训练过程中无法获取真实目标样本——即零样本仿真到现实迁移分割。我们聚焦于侧扫声纳图像中的沉船分割应用，提出新型分割网络STARS。该网络通过融合预测变形场与异常体体积，有效提升对真实声纳图像的泛化能力，实现更高效的图像分割零样本仿真到现实迁移。我们采用自主水下航行器（AUV）实地勘测获得的真实专家标注侧扫声纳沉船数据集，评估了方法的仿真到现实迁移能力。STARS完全在仿真环境中训练，无需在真实数据上进行任何微调即可实现零样本沉船分割。与最佳基线方法相比，本方法针对目标沉船类别的分割性能提升了20%。