In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics. In this paper, we show that if we cast a contrastive objective equivalently into the feature space, then its learning dynamics admits an interpretable form. Specifically, we show that its gradient descent corresponds to a specific message passing scheme on the corresponding augmentation graph. Based on this perspective, we theoretically characterize how contrastive learning gradually learns discriminative features with the alignment update and the uniformity update. Meanwhile, this perspective also establishes an intriguing connection between contrastive learning and Message Passing Graph Neural Networks (MP-GNNs). This connection not only provides a unified understanding of many techniques independently developed in each community, but also enables us to borrow techniques from MP-GNNs to design new contrastive learning variants, such as graph attention, graph rewiring, jumpy knowledge techniques, etc. We believe that our message passing perspective not only provides a new theoretical understanding of contrastive learning dynamics, but also bridges the two seemingly independent areas together, which could inspire more interleaving studies to benefit from each other. The code is available at https://github.com/PKU-ML/Message-Passing-Contrastive-Learning.

翻译：近年来，对比学习在自监督视觉表征学习领域取得了显著成果，但其学习动力学的严格理论解释仍显不足。本文证明，若将对比学习目标等价映射至特征空间，其学习动力学将呈现可解释形式。具体而言，梯度下降过程对应于相应增广图上的特定消息传递方案。基于这一视角，我们从理论上刻画了对比学习如何通过对齐更新和均匀性更新逐步学习判别性特征。同时，该视角还建立了对比学习与消息传递图神经网络（MP-GNNs）之间的有趣联系。这一关联不仅为两个领域独立发展的众多技术提供了统一理解框架，还使我们能够借鉴MP-GNN的技术设计新型对比学习变体，如图注意力、图重构、跳跃知识等技术。我们相信，这种消息传递视角不仅为对比学习动力学提供了新的理论认知，更将这两个看似独立的领域联系起来，有望激发更多交叉研究以相互促进。相关代码已开源至https://github.com/PKU-ML/Message-Passing-Contrastive-Learning。