Contrastive self-supervised learning based on point-wise comparisons has been widely studied for vision tasks. In the visual cortex of the brain, neuronal responses to distinct stimulus classes are organized into geometric structures known as neural manifolds. Accurate classification of stimuli can be achieved by effectively separating these manifolds, akin to solving a packing problem. We introduce Contrastive Learning As Manifold Packing (CLAMP), a self-supervised framework that recasts representation learning as a manifold packing problem. CLAMP introduces a loss function inspired by the potential energy of short-range repulsive particle systems, such as those encountered in the physics of simple liquids and jammed packings. In this framework, each class consists of sub-manifolds embedding multiple augmented views of a single image. The sizes and positions of the sub-manifolds are dynamically optimized by following the gradient of a packing loss. This approach yields interpretable dynamics in the embedding space that parallel jamming physics, and introduces geometrically meaningful hyperparameters within the loss function. Under the standard linear evaluation protocol, which freezes the backbone and trains only a linear classifier, CLAMP achieves competitive performance with state-of-the-art self-supervised models. Furthermore, our analysis reveals that neural manifolds corresponding to different categories emerge naturally and are effectively separated in the learned representation space, highlighting the potential of CLAMP to bridge insights from physics, neural science, and machine learning.

翻译：基于逐点比较的对比自监督学习在视觉任务中已被广泛研究。在大脑的视觉皮层中，神经元对不同刺激类别的响应被组织成称为神经流形的几何结构。通过有效分离这些流形可以实现对刺激的准确分类，这类似于解决一个填充问题。我们提出对比学习作为流形填充（CLAMP），这是一个将表示学习重新定义为流形填充问题的自监督框架。CLAMP引入了一种受短程排斥粒子系统（例如简单液体物理和阻塞填充中遇到的系统）势能启发的损失函数。在此框架中，每个类别包含嵌入单张图像多个增强视图的子流形。子流形的大小和位置通过遵循填充损失的梯度进行动态优化。这种方法在嵌入空间中产生了与阻塞物理相平行的可解释动力学，并在损失函数中引入了具有几何意义的超参数。在标准的线性评估协议下（即冻结主干网络，仅训练线性分类器），CLAMP实现了与最先进的自监督模型相竞争的性能。此外，我们的分析表明，对应于不同类别的神经流形在学习到的表示空间中自然涌现并被有效分离，这突显了CLAMP在连接物理学、神经科学和机器学习见解方面的潜力。