The effect of underrepresentation on the performance of minority groups is known to be a serious problem in supervised learning settings; however, it has been underexplored so far in the context of self-supervised learning (SSL). In this paper, we demonstrate that contrastive learning (CL), a popular variant of SSL, tends to collapse representations of minority groups with certain majority groups. We refer to this phenomenon as representation harm and demonstrate it on image and text datasets using the corresponding popular CL methods. Furthermore, our causal mediation analysis of allocation harm on a downstream classification task reveals that representation harm is partly responsible for it, thus emphasizing the importance of studying and mitigating representation harm. Finally, we provide a theoretical explanation for representation harm using a stochastic block model that leads to a representational neural collapse in a contrastive learning setting.

翻译：已知在监督学习场景中，少数群体的欠表征效应会严重影响其性能表现，然而这一现象在半监督学习（SSL）领域尚未得到充分研究。本文证明，作为半监督学习主流范式的对比学习（CL）倾向于将少数群体的表征与特定多数群体表征融合，我们将其命名为表征损害，并在图像与文本数据集上通过相应经典对比学习方法验证该现象的存在。进一步地，通过对下游分类任务的分配损害进行因果中介分析，我们发现表征损害是导致分配损害的部分原因，这凸显了研究并缓解表征损害的重要性。最终，我们采用随机块模型为对比学习场景中出现的表征性神经坍缩现象提供了理论解释。