Most state-of-the-art instance segmentation methods rely on large amounts of pixel-precise ground-truth annotations for training, which are expensive to create. Interactive segmentation networks help generate such annotations based on an image and the corresponding user interactions such as clicks. Existing methods for this task can only process a single instance at a time and each user interaction requires a full forward pass through the entire deep network. We introduce a more efficient approach, called DynaMITe, in which we represent user interactions as spatio-temporal queries to a Transformer decoder with a potential to segment multiple object instances in a single iteration. Our architecture also alleviates any need to re-compute image features during refinement, and requires fewer interactions for segmenting multiple instances in a single image when compared to other methods. DynaMITe achieves state-of-the-art results on multiple existing interactive segmentation benchmarks, and also on the new multi-instance benchmark that we propose in this paper.

翻译：当前最先进的实例分割方法大多依赖大量像素级精度的真实标注数据进行训练，而这类数据的标注成本高昂。交互式分割网络能够基于图像及相应的用户交互（如点击）生成此类标注。现有方法每次仅能处理单个实例，且每次用户交互需对整个深度网络执行完整前向传播。我们提出一种更高效的方法DynaMITe，将用户交互表示为Transformer解码器的时空查询，能够在单次迭代中分割多个目标实例。该架构无需在精化过程中重新计算图像特征，且相比其他方法，分割单张图像中多个实例所需的交互次数更少。DynaMITe在多个现有交互式分割基准测试及本文提出的新多实例基准测试中均取得了最优结果。