In this paper, we provide an information-theoretic perspective on Variance-Invariance-Covariance Regularization (VICReg) for self-supervised learning. To do so, we first demonstrate how information-theoretic quantities can be obtained for deterministic networks as an alternative to the commonly used unrealistic stochastic networks assumption. Next, we relate the VICReg objective to mutual information maximization and use it to highlight the underlying assumptions of the objective. Based on this relationship, we derive a generalization bound for VICReg, providing generalization guarantees for downstream supervised learning tasks and present new self-supervised learning methods, derived from a mutual information maximization objective, that outperform existing methods in terms of performance. This work provides a new information-theoretic perspective on self-supervised learning and Variance-Invariance-Covariance Regularization in particular and guides the way for improved transfer learning via information-theoretic self-supervised learning objectives.

翻译：在本文中,我们从信息理论角度审视差异-差异-差异-差异-常态化(VICREG),以便进行自我监督的学习;为此,我们首先展示如何为确定型网络获取信息理论数量,以替代通常使用的不切实际的随机网络假设;接着,我们将国际中心区域局的目标与相互信息最大化联系起来,并用它来突出目标的基本假设;根据这种关系,我们为维也纳国际中心区域局得出一个概括化,为下游受监督的学习任务提供一般化保障,并提出新的自我监督的学习方法,这些方法源自相互信息最大化目标,在业绩方面优于现有方法;这项工作特别为自我监督学习和差异-差异-差异-共变常态化提供了新的信息理论视角,并指导如何通过信息-理论自我监督的学习目标改进转让学习。</s>