In representation learning, regression has traditionally received less attention than classification. Directly applying representation learning techniques designed for classification to regression often results in fragmented representations in the latent space, yielding sub-optimal performance. In this paper, we argue that the potential of contrastive learning for regression has been overshadowed due to the neglect of two crucial aspects: ordinality-awareness and hardness. To address these challenges, we advocate "mixup your own contrastive pairs for supervised contrastive regression", instead of relying solely on real/augmented samples. Specifically, we propose Supervised Contrastive Learning for Regression with Mixup (SupReMix). It takes anchor-inclusive mixtures (mixup of the anchor and a distinct negative sample) as hard negative pairs and anchor-exclusive mixtures (mixup of two distinct negative samples) as hard positive pairs at the embedding level. This strategy formulates harder contrastive pairs by integrating richer ordinal information. Through extensive experiments on six regression datasets including 2D images, volumetric images, text, tabular data, and time-series signals, coupled with theoretical analysis, we demonstrate that SupReMix pre-training fosters continuous ordered representations of regression data, resulting in significant improvement in regression performance. Furthermore, SupReMix is superior to other approaches in a range of regression challenges including transfer learning, imbalanced training data, and scenarios with fewer training samples.

翻译：在表示学习中，回归任务相较于分类任务历来受到较少关注。直接将专为分类设计的表示学习技术应用于回归，往往会导致潜在空间中表示碎片化，进而产生次优性能。本文认为，对比学习在回归任务中的潜力因忽视了两个关键方面——序数感知性与困难度——而未被充分挖掘。为应对这些挑战，我们倡导"通过混合构造自有对比配对以实现有监督对比回归"，而非仅依赖真实或增强样本。具体而言，我们提出基于混合策略的回归有监督对比学习（SupReMix）。该方法在嵌入层面将包含锚点的混合样本（锚点与不同负样本的混合）作为困难负配对，将排除锚点的混合样本（两个不同负样本的混合）作为困难正配对。该策略通过整合更丰富的序数信息构建了更具挑战性的对比配对。通过在涵盖二维图像、立体图像、文本、表格数据及时序信号等六个回归数据集上的广泛实验与理论分析，我们证明SupReMix预训练能够促进回归数据形成连续有序的表示，从而显著提升回归性能。此外，在迁移学习、非平衡训练数据及小样本场景等回归挑战中，SupReMix均优于其他方法。