Deep spectral methods reframe the image decomposition process as a graph partitioning task by extracting features using self-supervised learning and utilizing the Laplacian of the affinity matrix to obtain eigensegments. However, instance segmentation has received less attention compared to other tasks within the context of deep spectral methods. This paper addresses the fact that not all channels of the feature map extracted from a self-supervised backbone contain sufficient information for instance segmentation purposes. In fact, Some channels are noisy and hinder the accuracy of the task. To overcome this issue, this paper proposes two channel reduction modules: Noise Channel Reduction (NCR) and Deviation-based Channel Reduction (DCR). The NCR retains channels with lower entropy, as they are less likely to be noisy, while DCR prunes channels with low standard deviation, as they lack sufficient information for effective instance segmentation. Furthermore, the paper demonstrates that the dot product, commonly used in deep spectral methods, is not suitable for instance segmentation due to its sensitivity to feature map values, potentially leading to incorrect instance segments. A new similarity metric called Bray-Curtis over Chebyshev (BoC) is proposed to address this issue. It takes into account the distribution of features in addition to their values, providing a more robust similarity measure for instance segmentation. Quantitative and qualitative results on the Youtube-VIS2019 dataset highlight the improvements achieved by the proposed channel reduction methods and the use of BoC instead of the conventional dot product for creating the affinity matrix. These improvements are observed in terms of mean Intersection over Union and extracted instance segments, demonstrating enhanced instance segmentation performance. The code is available on: https://github.com/farnooshar/SpecUnIIS

翻译：深度谱方法通过自监督学习提取特征，并利用亲和矩阵的拉普拉斯算子获取特征分割（eigensegments），将图像分解过程重构为图划分任务。然而，在深度谱方法的研究背景下，实例分割相较于其他任务受到的关注较少。本文指出，从自监督骨干网络提取的特征图中，并非所有通道都包含足以支撑实例分割的信息。事实上，部分通道存在噪声，会阻碍任务精度的提升。为解决此问题，本文提出两种通道缩减模块：噪声通道缩减（NCR）与偏差驱动通道缩减（DCR）。NCR保留熵值较低的通道（因其噪声可能性更小），而DCR则剔除标准差低的通道（因其缺乏有效实例分割所需的充分信息）。此外，本文证明了深度谱方法中常用的点积运算因对特征图数值敏感，可能导致错误的实例分割结果，故不适用于该任务。为此，提出一种名为"基于切比雪夫的布雷-柯蒂斯相似度（BoC）"的新型相似度度量，该度量在数值之外同时考虑特征分布，为实例分割提供更鲁棒的相似性指标。在Youtube-VIS2019数据集上的定量与定性结果表明，所提出的通道缩减方法以及采用BoC替代传统点积构建亲和矩阵的方案，在平均交并比与提取的实例分割片段方面均实现了性能提升。代码已开源至：https://github.com/farnooshar/SpecUnIIS