Recent advances in self-supervised representation learning have enabled more efficient and robust model performance without relying on extensive labeled data. However, most works are still focused on images, with few working on videos and even fewer on multi-view videos, where more powerful inductive biases can be leveraged for self-supervision. In this work, we propose a novel method for representation learning of multi-view videos, where we explicitly model the representation space to maintain Homography Equivariance (HomE). Our method learns an implicit mapping between different views, culminating in a representation space that maintains the homography relationship between neighboring views. We evaluate our HomE representation via action recognition and pedestrian intent prediction as downstream tasks. On action classification, our method obtains 96.4% 3-fold accuracy on the UCF101 dataset, better than most state-of-the-art self-supervised learning methods. Similarly, on the STIP dataset, we outperform the state-of-the-art by 6% for pedestrian intent prediction one second into the future while also obtaining an accuracy of 91.2% for pedestrian action (cross vs. not-cross) classification. Code is available at https://github.com/anirudhs123/HomE.

翻译：近期自监督表示学习的进展使得模型在无需依赖大量标注数据的情况下，能够实现更高效、更鲁棒的性能。然而，大多数研究仍聚焦于图像领域，少数工作涉及视频，而针对多视角视频的研究则更为稀少——此类场景可利用更强的归纳偏置来促进自监督学习。本文提出一种新颖的多视角视频表示学习方法，通过显式建模表示空间以维持单应性等变性（HomE）。该方法学习不同视角间的隐式映射，最终构建出能保持相邻视角单应关系的表示空间。我们通过动作识别和行人意图预测两项下游任务评估HomE表示的性能。在动作分类任务中，本方法在UCF101数据集上取得96.4%的三折准确率，优于多数最先进的自监督学习方法。类似地，在STIP数据集上，本方法对未来一秒的行人意图预测相比现有最优方法提升6%，同时行人动作（过街与不过街）分类准确率达到91.2%。相关代码已开源至https://github.com/anirudhs123/HomE。