The discrimination of instance embeddings plays a vital role in associating instances across time for online video instance segmentation (VIS). Instance embedding learning is directly supervised by the contrastive loss computed upon the contrastive items (CIs), which are sets of anchor/positive/negative embeddings. Recent online VIS methods leverage CIs sourced from one reference frame only, which we argue is insufficient for learning highly discriminative embeddings. Intuitively, a possible strategy to enhance CIs is replicating the inference phase during training. To this end, we propose a simple yet effective training strategy, called Consistent Training for Online VIS (CTVIS), which devotes to aligning the training and inference pipelines in terms of building CIs. Specifically, CTVIS constructs CIs by referring inference the momentum-averaged embedding and the memory bank storage mechanisms, and adding noise to the relevant embeddings. Such an extension allows a reliable comparison between embeddings of current instances and the stable representations of historical instances, thereby conferring an advantage in modeling VIS challenges such as occlusion, re-identification, and deformation. Empirically, CTVIS outstrips the SOTA VIS models by up to +5.0 points on three VIS benchmarks, including YTVIS19 (55.1% AP), YTVIS21 (50.1% AP) and OVIS (35.5% AP). Furthermore, we find that pseudo-videos transformed from images can train robust models surpassing fully-supervised ones.

翻译：实例嵌入的判别性对于在线视频实例分割（VIS）中跨时间关联实例至关重要。实例嵌入学习直接依赖于对比项（CIs）计算的对比损失进行监督，这些对比项由锚点/正例/负例嵌入集构成。当前在线VIS方法仅使用从单一参考帧获取的对比项，我们论证这种做法不足以学习高判别性嵌入。直观而言，增强对比项的一种可行策略是在训练阶段复现推理过程。为此，我们提出一种简单有效的训练策略——面向在线VIS的一致性训练（CTVIS），致力于在构建对比项时对齐训练与推理流程。具体而言，CTVIS通过引用动量平均嵌入与记忆库存储机制的推理过程构建对比项，并对相关嵌入添加噪声。这种扩展使得当前实例嵌入与历史实例稳定表征之间能够进行可靠比较，从而在应对遮挡、重识别和形变等VIS挑战时获得优势。实验表明，CTVIS在YTVIS19（55.1% AP）、YTVIS21（50.1% AP）和OVIS（35.5% AP）三个VIS基准上较现有最优VIS模型提升高达+5.0个点。此外，我们发现由图像转换的伪视频可训练出超越全监督方法的鲁棒模型。