In recent years, the development of instance segmentation has garnered significant attention in a wide range of applications. However, the training of a fully-supervised instance segmentation model requires costly both instance-level and pixel-level annotations. In contrast, weakly-supervised instance segmentation methods (i.e., with image-level class labels or point labels) struggle to satisfy the accuracy and recall requirements of practical scenarios. In this paper, we propose a novel paradigm called synthetic instance segmentation (SISeg), which achieves Instance Segmentation results from image masks predicted using off-the-shelf semantic segmentation models. SISeg does not require training a semantic or/and instance segmentation model and avoids the need for instance-level image annotations. Therefore, it is highly efficient. Specifically, we first obtain a semantic segmentation mask of the input image via a trained semantic segmentation model. Then, we calculate a displacement field vector for each pixel based on the segmentation mask, which can indicate representations belonging to the same class but different instances, i.e., obtaining the instance-level object information. Finally, instance segmentation results are obtained after being refined by a learnable category-agnostic object boundary branch. Extensive experimental results on two challenging datasets and representative semantic segmentation baselines (including CNNs and Transformers) demonstrate that SISeg can achieve competitive results compared to the state-of-the-art fully-supervised instance segmentation methods without the need for additional human resources or increased computational costs. The code is available at: SISeg

翻译：近年来，实例分割技术在各种应用场景中引起了广泛关注。然而，全监督实例分割模型的训练需要昂贵的实例级和像素级标注。相比之下，弱监督实例分割方法（如使用图像级类别标签或点标签）难以满足实际场景中的精度和召回率要求。本文提出了一种名为合成实例分割的新范式，该方法利用现成语义分割模型预测的图像掩码来实现实例分割结果。SISeg无需训练语义或/和实例分割模型，也避免了实例级图像标注的需求，因此具有极高的效率。具体而言，我们首先通过训练好的语义分割模型获取输入图像的语义分割掩码；然后基于该掩码计算每个像素的位移场向量，该向量可表示属于同一类别但不同实例的表征，从而获取实例级对象信息；最后通过可学习的类别无关对象边界分支进行细化，得到实例分割结果。在两个具有挑战性的数据集和代表性语义分割基线（包括CNN和Transformer）上的大量实验结果表明，SISeg无需额外人力资源或增加计算成本即可达到与最先进的全监督实例分割方法相媲美的结果。代码已开源：SISeg