Video instance segmentation (VIS) is a critical task with diverse applications, including autonomous driving and video editing. Existing methods often underperform on complex and long videos in real world, primarily due to two factors. Firstly, offline methods are limited by the tightly-coupled modeling paradigm, which treats all frames equally and disregards the interdependencies between adjacent frames. Consequently, this leads to the introduction of excessive noise during long-term temporal alignment. Secondly, online methods suffer from inadequate utilization of temporal information. To tackle these challenges, we propose a decoupling strategy for VIS by dividing it into three independent sub-tasks: segmentation, tracking, and refinement. The efficacy of the decoupling strategy relies on two crucial elements: 1) attaining precise long-term alignment outcomes via frame-by-frame association during tracking, and 2) the effective utilization of temporal information predicated on the aforementioned accurate alignment outcomes during refinement. We introduce a novel referring tracker and temporal refiner to construct the \textbf{D}ecoupled \textbf{VIS} framework (\textbf{DVIS}). DVIS achieves new SOTA performance in both VIS and VPS, surpassing the current SOTA methods by 7.3 AP and 9.6 VPQ on the OVIS and VIPSeg datasets, which are the most challenging and realistic benchmarks. Moreover, thanks to the decoupling strategy, the referring tracker and temporal refiner are super light-weight (only 1.69\% of the segmenter FLOPs), allowing for efficient training and inference on a single GPU with 11G memory. The code is available at \href{https://github.com/zhang-tao-whu/DVIS}{https://github.com/zhang-tao-whu/DVIS}.

翻译：视频实例分割（VIS）是一项关键任务，具有广泛的应用场景，包括自动驾驶和视频编辑。现有方法在现实世界中复杂的长视频上往往表现不佳，主要归因于两个因素。首先，离线方法受限于紧耦合建模范式，该范式平等处理所有帧而忽视相邻帧之间的相互依赖关系，导致在长期时间对齐过程中引入过多噪声。其次，在线方法对时间信息的利用不充分。为解决这些挑战，我们提出一种针对VIS的解耦策略，将其划分为三个独立子任务：分割、跟踪和精炼。该解耦策略的有效性依赖于两个关键要素：1）在跟踪过程中通过逐帧关联实现精确的长期对齐结果；2）在精炼过程中基于前述精确对齐结果有效利用时间信息。我们引入新型参考跟踪器和时间精炼器，构建了解耦式VIS框架（DVIS）。DVIS在VIS和VPS任务中均达到新的最优性能，在最具挑战性和真实性的基准数据集OVIS和VIPSeg上，分别超越现有最优方法7.3 AP和9.6 VPQ。此外，得益于解耦策略，参考跟踪器和时间精炼器极为轻量化（仅占分割器FLOPs的1.69%），使得在11G显存的单GPU上进行高效训练和推理成为可能。代码已开源至：https://github.com/zhang-tao-whu/DVIS。