This paper presents a novel method for face clustering in videos using a video-centralised transformer. Previous works often employed contrastive learning to learn frame-level representation and used average pooling to aggregate the features along the temporal dimension. This approach may not fully capture the complicated video dynamics. In addition, despite the recent progress in video-based contrastive learning, few have attempted to learn a self-supervised clustering-friendly face representation that benefits the video face clustering task. To overcome these limitations, our method employs a transformer to directly learn video-level representations that can better reflect the temporally-varying property of faces in videos, while we also propose a video-centralised self-supervised framework to train the transformer model. We also investigate face clustering in egocentric videos, a fast-emerging field that has not been studied yet in works related to face clustering. To this end, we present and release the first large-scale egocentric video face clustering dataset named EasyCom-Clustering. We evaluate our proposed method on both the widely used Big Bang Theory (BBT) dataset and the new EasyCom-Clustering dataset. Results show the performance of our video-centralised transformer has surpassed all previous state-of-the-art methods on both benchmarks, exhibiting a self-attentive understanding of face videos.

翻译：本文提出了一种利用视频中心Transformer进行视频人脸聚类的新方法。以往的工作通常采用对比学习来学习帧级表示，并使用平均池化沿时间维度聚合特征。这种方法可能无法充分捕捉复杂的视频动态。此外，尽管基于视频的对比学习近期取得了进展，但很少有研究尝试学习一种有助于视频人脸聚类的自监督聚类友好型人脸表示。为克服这些限制，我们的方法采用Transformer直接学习能够更好反映视频中人脸时变特性的视频级表示，同时我们提出了一种视频中心化的自监督框架来训练Transformer模型。我们还研究了自我中心视频中的人脸聚类——这是一个快速兴起的领域，尚未在与人脸聚类相关的工作中得到研究。为此，我们提出并发布了首个大规模自我中心视频人脸聚类数据集EasyCom-Clustering。我们在广泛使用的《生活大爆炸》（BBT）数据集和新的EasyCom-Clustering数据集上评估了所提出的方法。结果表明，我们的视频中心Transformer在两个基准测试上的性能均超越了所有先前的最佳方法，展现了对人脸视频的自注意力理解。