Unsupervised feature learning has made great strides with contrastive learning based on instance discrimination and invariant mapping, as benchmarked on curated class-balanced datasets. However, natural data could be highly correlated and long-tail distributed. Natural between-instance similarity conflicts with the presumed instance distinction, causing unstable training and poor performance. Our idea is to discover and integrate between-instance similarity into contrastive learning, not directly by instance grouping, but by cross-level discrimination (CLD) between instances and local instance groups. While invariant mapping of each instance is imposed by attraction within its augmented views, between-instance similarity emerges from common repulsion against instance groups. Our batch-wise and cross-view comparisons also greatly improve the positive/negative sample ratio of contrastive learning and achieve better invariant mapping. To effect both grouping and discrimination objectives, we impose them on features separately derived from a shared representation. In addition, we propose normalized projection heads and unsupervised hyper-parameter tuning for the first time. Our extensive experimentation demonstrates that CLD is a lean and powerful add-on to existing methods (e.g., NPID, MoCo, InfoMin, BYOL) on highly correlated, long-tail, or balanced datasets. It not only achieves new state-of-the-art on self-supervision, semi-supervision, and transfer learning benchmarks, but also beats MoCo v2 and SimCLR on every reported performance attained with a much larger compute. CLD effectively extends unsupervised learning to natural data and brings it closer to real-world applications.

翻译：未经监督的地物学习取得了长足的进步,通过基于实例歧视的对比性学习和以分类平衡的类比数据集为基准的无差异绘图,取得了巨大的进步;然而,自然数据可能高度相关,而且分布时间长; 自然内部的相似性与假定实例的区别发生自然的相似性冲突,造成培训不稳定和业绩差。 我们的想法是发现和整合从内部的相似性,形成对比性学习,而不是直接通过实例分组,而是通过实例与地方实例群体之间的跨层次歧视(CLD) 。虽然每个实例的不变化性格绘图是通过在其扩大的观点中吸引的,而使内部的相似性能从普通的对实例组的反常态中出现。 我们的批量和交叉视图比较还大大改进了对比性学习的积极/负性抽样比率,并取得了更好的内向性图。 为了实现组合和歧视目标,我们把它们分别放在从共同代表制中分离出来的特征上。 此外,我们提出了标准化的投影头和不精准性超常数的超值应用调。 我们的广泛实验表明,Simal-和强力的内置-IG-IF-IG-ID-ID-ID-I-I-IL-ID-I-I-G-ID-G-ID-IL-ID-G-G-G-G-G-G-G-G-ID-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-ID-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-G-