Uniformity plays a crucial role in the assessment of learned representations, contributing to a deeper comprehension of self-supervised learning. The seminal work by \citet{Wang2020UnderstandingCR} introduced a uniformity metric that quantitatively measures the collapse degree of learned representations. Directly optimizing this metric together with alignment proves to be effective in preventing constant collapse. However, we present both theoretical and empirical evidence revealing that this metric lacks sensitivity to dimensional collapse, highlighting its limitations. To address this limitation and design a more effective uniformity metric, this paper identifies five fundamental properties, some of which the existing uniformity metric fails to meet. We subsequently introduce a novel uniformity metric that satisfies all of these desiderata and exhibits sensitivity to dimensional collapse. When applied as an auxiliary loss in various established self-supervised methods, our proposed uniformity metric consistently enhances their performance in downstream tasks.Our code was released at https://github.com/sunset-clouds/WassersteinUniformityMetric.

翻译：均匀性在评估学习表示中起着关键作用，有助于更深入地理解自监督学习。Wang等人的开创性工作（Wang等，2020）引入了一种均匀性度量，用于定量衡量学习表示的坍缩程度。直接将此度量与对齐一起优化被证明能有效防止恒定坍缩。然而，我们提出理论和实验证据表明，该度量对维度坍缩缺乏敏感性，揭示了其局限性。为解决这一局限并设计更有效的均匀性度量，本文识别出五个基本性质，其中部分性质现有均匀性度量未能满足。我们随后引入一种新的均匀性度量，该度量满足所有这些期望属性，并对维度坍缩具有敏感性。当将其作为辅助损失应用于各种已有自监督方法时，我们提出的均匀性度量持续提升了它们在后续任务中的性能。我们的代码已发布在https://github.com/sunset-clouds/WassersteinUniformityMetric。