The goal of video segmentation is to accurately segment and track every pixel in diverse scenarios. In this paper, we present Tube-Link, a versatile framework that addresses multiple core tasks of video segmentation with a unified architecture. Our framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks. To enhance the modeling of cross-tube relationships, we propose an effective way to perform tube-level linking via attention along the queries. In addition, we introduce temporal contrastive learning to instance-wise discriminative features for tube-level association. Our approach offers flexibility and efficiency for both short and long video inputs, as the length of each subclip can be varied according to the needs of datasets or scenarios. Tube-Link outperforms existing specialized architectures by a significant margin on five video segmentation datasets. Specifically, it achieves almost 13% relative improvements on VIPSeg and 4% improvements on KITTI-STEP over the strong baseline Video K-Net. When using a ResNet50 backbone on Youtube-VIS-2019 and 2021, Tube-Link boosts IDOL by 3% and 4%, respectively. Code will be available.

翻译：视频分割的目标是在多样化场景中精确分割并追踪每一个像素。本文提出Tube-Link——一个以统一架构解决多项视频分割核心任务的通用框架。该框架采用近在线方法，通过输入短视频片段输出对应的时空管状掩码。为增强跨管关系建模，我们提出了一种有效方法，通过沿查询维度的注意力机制实现管级关联。此外，我们引入时序对比学习以增强实例级判别特征，促进管级关联。本方法能灵活高效地处理长短视频输入——每个子片段的长度可根据数据集或场景需求调整。Tube-Link在五个视频分割数据集上显著超越现有专用架构：具体而言，其在VIPSeg上相较强基线Video K-Net获得近13%的相对提升，在KITTI-STEP上提升4%；当采用ResNet50骨干网络时，Tube-Link在Youtube-VIS-2019和2021上分别比IDOL提升3%和4%。代码将开源。