Most state-of-the-art instance segmentation methods rely on large amounts of pixel-precise ground-truth annotations for training, which are expensive to create. Interactive segmentation networks help generate such annotations based on an image and the corresponding user interactions such as clicks. Existing methods for this task can only process a single instance at a time and each user interaction requires a full forward pass through the entire deep network. We introduce a more efficient approach, called DynaMITe, in which we represent user interactions as spatio-temporal queries to a Transformer decoder with a potential to segment multiple object instances in a single iteration. Our architecture also alleviates any need to re-compute image features during refinement, and requires fewer interactions for segmenting multiple instances in a single image when compared to other methods. DynaMITe achieves state-of-the-art results on multiple existing interactive segmentation benchmarks, and also on the new multi-instance benchmark that we propose in this paper.

翻译：目前最先进的实例分割方法大多依赖大量像素级精确标注的真实数据进行训练，而此类标注的生成成本高昂。交互式分割网络能够基于图像及用户点击等交互行为生成此类标注。现有方法仅能逐实例处理，且每次用户交互都需要对整个深度网络执行完整的前向传播。我们提出了一种更高效的名为DynaMITe的方法，该方法将用户交互表示为应用于Transformer解码器的时空查询，能够在单次迭代中分割多个目标实例。本架构免除了优化过程中重新计算图像特征的需求，且与现有方法相比，对单张图像中的多实例分割所需交互次数更少。DynaMITe在多个现有交互式分割基准测试以及本文提出的新多实例基准测试中均达到最先进水平。