Unsupervised Contrastive learning has gained prominence in fields such as vision, and biology, leveraging predefined positive/negative samples for representation learning. Data augmentation, categorized into hand-designed and model-based methods, has been identified as a crucial component for enhancing contrastive learning. However, hand-designed methods require human expertise in domain-specific data while sometimes distorting the meaning of the data. In contrast, generative model-based approaches usually require supervised or large-scale external data, which has become a bottleneck constraining model training in many domains. To address the problems presented above, this paper proposes DiffAug, a novel unsupervised contrastive learning technique with diffusion mode-based positive data generation. DiffAug consists of a semantic encoder and a conditional diffusion model; the conditional diffusion model generates new positive samples conditioned on the semantic encoding to serve the training of unsupervised contrast learning. With the help of iterative training of the semantic encoder and diffusion model, DiffAug improves the representation ability in an uninterrupted and unsupervised manner. Experimental evaluations show that DiffAug outperforms hand-designed and SOTA model-based augmentation methods on DNA sequence, visual, and bio-feature datasets. The code for review is released at \url{https://github.com/zangzelin/code_diffaug}.
翻译:无监督对比学习在视觉和生物学等领域日益受到重视,它利用预定义的正/负样本进行表示学习。数据增强分为人工设计和基于模型的方法,已被认为是增强对比学习的关键组成部分。然而,人工设计方法需要领域特定数据的人类专业知识,有时会扭曲数据的含义。相比之下,基于生成模型的方法通常需要监督或大规模外部数据,这已成为限制许多领域模型训练的瓶颈。为解决上述问题,本文提出DiffAug,一种基于扩散模式的正样本生成的新型无监督对比学习技术。DiffAug由语义编码器和条件扩散模型组成;条件扩散模型根据语义编码生成新的正样本,以服务于无监督对比学习的训练。借助语义编码器和扩散模型的迭代训练,DiffAug以连续且无监督的方式提高了表示能力。实验评估表明,在DNA序列、视觉和生物特征数据集上,DiffAug优于人工设计和基于SOTA模型的增强方法。审查代码发布于\url{https://github.com/zangzelin/code_diffaug}。