This paper describes our champion solution for the CVPR2022 Generic Event Boundary Captioning (GEBC) competition. GEBC requires the captioning model to have a comprehension of instantaneous status changes around the given video boundary, which makes it much more challenging than conventional video captioning task. In this paper, a Dual-Stream Transformer with improvements on both video content encoding and captions generation is proposed: (1) We utilize three pre-trained models to extract the video features from different granularities. Moreover, we exploit the types of boundary as hints to help the model generate captions. (2) We particularly design an model, termed as Dual-Stream Transformer, to learn discriminative representations for boundary captioning. (3) Towards generating content-relevant and human-like captions, we improve the description quality by designing a word-level ensemble strategy. The promising results on the GEBC test split demonstrate the efficacy of our proposed model.

翻译：本文描述了我们在CVPR2022通用事件边界字幕生成（GEBC）竞赛中的冠军解决方案。GEBC要求字幕生成模型能够理解给定视频边界周围的瞬时状态变化，这使得其比传统视频字幕生成任务更具挑战性。本文提出了一种双流Transformer模型，该模型在视频内容编码和字幕生成方面均进行了改进：（1）我们利用三个预训练模型从不同粒度提取视频特征。此外，我们利用边界类型作为提示信息来帮助模型生成字幕。（2）我们特别设计了一种名为双流Transformer的模型，用于学习边界字幕生成的判别性表示。（3）为了生成内容相关且类人的字幕，我们通过设计词级集成策略来提高描述质量。在GEBC测试集上的优异结果证明了我们提出模型的有效性。