Video instance segmentation (VIS) is a critical task with diverse applications, including autonomous driving and video editing. Existing methods often underperform on complex and long videos in real world, primarily due to two factors. Firstly, offline methods are limited by the tightly-coupled modeling paradigm, which treats all frames equally and disregards the interdependencies between adjacent frames. Consequently, this leads to the introduction of excessive noise during long-term temporal alignment. Secondly, online methods suffer from inadequate utilization of temporal information. To tackle these challenges, we propose a decoupling strategy for VIS by dividing it into three independent sub-tasks: segmentation, tracking, and refinement. The efficacy of the decoupling strategy relies on two crucial elements: 1) attaining precise long-term alignment outcomes via frame-by-frame association during tracking, and 2) the effective utilization of temporal information predicated on the aforementioned accurate alignment outcomes during refinement. We introduce a novel referring tracker and temporal refiner to construct the \textbf{D}ecoupled \textbf{VIS} framework (\textbf{DVIS}). DVIS achieves new SOTA performance in both VIS and VPS, surpassing the current SOTA methods by 7.3 AP and 9.6 VPQ on the OVIS and VIPSeg datasets, which are the most challenging and realistic benchmarks. Moreover, thanks to the decoupling strategy, the referring tracker and temporal refiner are super light-weight (only 1.69\% of the segmenter FLOPs), allowing for efficient training and inference on a single GPU with 11G memory. The code is available at \href{https://github.com/zhang-tao-whu/DVIS}{https://github.com/zhang-tao-whu/DVIS}.

翻译：视频实例分割（VIS）是一项关键任务，在自动驾驶和视频编辑等众多领域具有广泛应用。现有方法在现实世界中复杂且冗长的视频上表现不佳，主要原因有二。首先，离线方法受限于紧耦合建模范式，该范式平等处理所有帧而忽视相邻帧间的相互依赖关系，导致长期时序对齐时引入过多噪声。其次，在线方法存在时序信息利用不充分的问题。为应对这些挑战，我们提出了一种针对VIS的解耦策略，将其划分为三个独立的子任务：分割、跟踪与精炼。该解耦策略的有效性依赖于两个关键要素：1）在跟踪过程中通过逐帧关联获得精确的长期对齐结果；2）在精炼阶段基于上述精确对齐结果有效利用时序信息。我们引入新颖的引用跟踪器与时序精炼器，构建了解耦式VIS框架（DVIS）。DVIS在VIS与VPS任务中均达到新的SOTA性能，在最具挑战性和现实性的基准数据集OVIS与VIPSeg上，分别超越当前最先进方法7.3 AP和9.6 VPQ。此外，得益于解耦策略，引用跟踪器与时序精炼器极其轻量（仅占分割器FLOPs的1.69%），使得在11G显存的单GPU上即可实现高效训练与推理。代码已开源在https://github.com/zhang-tao-whu/DVIS。