This paper presents a novel method for face clustering in videos using a video-centralised transformer. Previous works often employed contrastive learning to learn frame-level representation and used average pooling to aggregate the features along the temporal dimension. This approach may not fully capture the complicated video dynamics. In addition, despite the recent progress in video-based contrastive learning, few have attempted to learn a self-supervised clustering-friendly face representation that benefits the video face clustering task. To overcome these limitations, our method employs a transformer to directly learn video-level representations that can better reflect the temporally-varying property of faces in videos, while we also propose a video-centralised self-supervised framework to train the transformer model. We also investigate face clustering in egocentric videos, a fast-emerging field that has not been studied yet in works related to face clustering. To this end, we present and release the first large-scale egocentric video face clustering dataset named EasyCom-Clustering. We evaluate our proposed method on both the widely used Big Bang Theory (BBT) dataset and the new EasyCom-Clustering dataset. Results show the performance of our video-centralised transformer has surpassed all previous state-of-the-art methods on both benchmarks, exhibiting a self-attentive understanding of face videos.

翻译：本文提出了一种新颖的视频人脸聚类方法，采用视频中心化Transformer。此前的研究通常利用对比学习学习帧级表示，并使用平均池化沿时间维度聚合特征。这种方法可能无法充分捕捉复杂的视频动态信息。此外，尽管基于视频的对比学习近年取得进展，但鲜有研究尝试学习有利于视频人脸聚类任务的自监督聚类友好型人脸表示。为克服这些局限，我们的方法采用Transformer直接学习能更好反映视频中人脸时变特性的视频级表示，同时提出一个视频中心化自监督框架来训练Transformer模型。我们还针对快速兴起的自我中心视频中的人脸聚类问题进行了研究——这一方向在人脸聚类相关工作中尚未涉及。为此，我们创建并发布了首个大规模自我中心视频人脸聚类数据集EasyCom-Clustering。在广泛使用的《生活大爆炸》（BBT）数据集和新提出的EasyCom-Clustering数据集上的评估结果表明，我们的视频中心化Transformer在两个基准测试中均超越了以往所有最先进方法，展现出对人脸视频的自主注意力理解能力。