Recent progress in interactive point prompt based Image Segmentation allows to significantly reduce the manual effort to obtain high quality semantic labels. State-of-the-art unsupervised methods use self-supervised pre-trained models to obtain pseudo-labels which are used in training a prompt-based segmentation model. In this paper, we propose a novel unsupervised and training-free approach based solely on the self-attention of Stable Diffusion. We interpret the self-attention tensor as a Markov transition operator, which enables us to iteratively construct a Markov chain. Pixel-wise counting of the required number of iterations along the Markov-chain to reach a relative probability threshold yields a Markov-iteration-map, which we simply call a Markov-map. Compared to the raw attention maps, we show that our proposed Markov-map has less noise, sharper semantic boundaries and more uniform values within semantically similar regions. We integrate the Markov-map in a simple yet effective truncated nearest neighbor framework to obtain interactive point prompt based segmentation. Despite being training-free, we experimentally show that our approach yields excellent results in terms of Number of Clicks (NoC), even outperforming state-of-the-art training based unsupervised methods in most of the datasets.

翻译：基于交互式点提示的图像分割技术近期取得显著进展，大幅降低了获取高质量语义标注所需的人工成本。当前最先进的无监督方法采用自监督预训练模型生成伪标签，进而训练基于提示的分割模型。本文提出一种全新的无监督且免训练的方法，该方法仅利用Stable Diffusion的自注意力机制。我们将自注意力张量解释为马尔可夫转移算子，从而能够迭代构建马尔可夫链。通过沿马尔可夫链统计像素达到相对概率阈值所需的迭代次数，得到马尔可夫迭代图（简称为马尔可夫图）。相较于原始注意力图，我们证明所提出的马尔可夫图具有更低的噪声、更清晰的语义边界以及在语义相似区域内更均匀的数值分布。我们将马尔可夫图集成到一种简洁高效的截断最近邻框架中，实现基于交互式点提示的分割。尽管完全免训练，实验结果表明我们的方法在点击次数（NoC）指标上表现优异，在多数数据集上甚至超越了当前最先进的基于训练的无监督方法。