Accurate segmentation of polyps from colonoscopy videos is of great significance to polyp treatment and early prevention of colorectal cancer. However, it is challenging due to the difficulties associated with modelling long-range spatio-temporal relationships within a colonoscopy video. In this paper, we address this challenging task with a novel Mixture-Attention Siamese Transformer (MAST), which explicitly models the long-range spatio-temporal relationships with a mixture-attention mechanism for accurate polyp segmentation. Specifically, we first construct a Siamese transformer architecture to jointly encode paired video frames for their feature representations. We then design a mixture-attention module to exploit the intra-frame and inter-frame correlations, enhancing the features with rich spatio-temporal relationships. Finally, the enhanced features are fed to two parallel decoders for predicting the segmentation maps. To the best of our knowledge, our MAST is the first transformer model dedicated to video polyp segmentation. Extensive experiments on the large-scale SUN-SEG benchmark demonstrate the superior performance of MAST in comparison with the cutting-edge competitors. Our code is publicly available at https://github.com/Junqing-Yang/MAST.

翻译：从结肠镜视频中精确分割息肉对息肉治疗和结直肠癌的早期预防具有重要意义。然而，由于结肠镜视频中长程时空关系建模的困难性，这一任务具有挑战性。本文提出了一种新颖的混合注意力孪生Transformer（MAST）来应对这一挑战性任务，该模型通过混合注意力机制显式建模长程时空关系，实现精确的息肉分割。具体而言，我们首先构建孪生Transformer架构以联合编码成对视频帧的特征表示；接着设计混合注意力模块来挖掘帧内与帧间相关性，通过丰富的时空关系增强特征；最后将增强后的特征输入两个并行解码器以预测分割图。据我们所知，MAST是首个专用于视频息肉分割的Transformer模型。在大型SUN-SEG基准上的大量实验表明，MAST相较于最先进的竞争方法具有优越性能。我们的代码已开源在https://github.com/Junqing-Yang/MAST。