Interactive image segmentation enables annotators to efficiently perform pixel-level annotation for segmentation tasks. However, the existing interactive segmentation pipeline suffers from inefficient computations of interactive models because of the following two issues. First, annotators' later click is based on models' feedback of annotators' former click. This serial interaction is unable to utilize model's parallelism capabilities. Second, the model has to repeatedly process the image, the annotator's current click, and the model's feedback of the annotator's former clicks at each step of interaction, resulting in redundant computations. For efficient computation, we propose a method named InterFormer that follows a new pipeline to address these issues. InterFormer extracts and preprocesses the computationally time-consuming part i.e. image processing from the existing process. Specifically, InterFormer employs a large vision transformer (ViT) on high-performance devices to preprocess images in parallel, and then uses a lightweight module called interactive multi-head self attention (I-MSA) for interactive segmentation. Furthermore, the I-MSA module's deployment on low-power devices extends the practical application of interactive segmentation. The I-MSA module utilizes the preprocessed features to efficiently response to the annotator inputs in real-time. The experiments on several datasets demonstrate the effectiveness of InterFormer, which outperforms previous interactive segmentation models in terms of computational efficiency and segmentation quality, achieve real-time high-quality interactive segmentation on CPU-only devices.

翻译：[translated abstract in Chinese] 交互式图像分割技术使得标注人员能够高效地完成分割任务的像素级标注。然而，现有交互式分割流程因以下两个问题存在交互模型计算效率低下的缺陷。其一，标注者的后续点击依赖于模型对前次点击的反馈，这种串行交互方式无法充分利用模型的并行计算能力。其二，模型在每次交互步骤中都需要重复处理图像、标注者当前点击以及模型对前次点击的反馈，导致大量冗余计算。为实现高效计算，我们提出了一种名为InterFormer的新方法，该方法采用新流程来解决上述问题。InterFormer将计算耗时部分（即图像处理）从现有流程中剥离并进行预处理。具体而言，InterFormer在高性能设备上采用大型视觉Transformer（ViT）对图像进行并行预处理，随后使用名为交互式多头自注意力（I-MSA）的轻量级模块执行交互式分割。此外，I-MSA模块在低功耗设备上的部署拓展了交互式分割的实际应用场景。该模块利用预处理后的特征实时高效响应标注者的输入操作。在多个数据集上的实验表明，InterFormer在计算效率和分割质量上均优于现有交互式分割模型，能够在仅搭载CPU的设备上实现实时高质量交互式分割。