We introduce DNABERT-S, a tailored genome model that develops species-aware embeddings to naturally cluster and segregate DNA sequences of different species in the embedding space. Differentiating species from genomic sequences (i.e., DNA and RNA) is vital yet challenging, since many real-world species remain uncharacterized, lacking known genomes for reference. Embedding-based methods are therefore used to differentiate species in an unsupervised manner. DNABERT-S builds upon a pre-trained genome foundation model named DNABERT-2. To encourage effective embeddings to error-prone long-read DNA sequences, we introduce Manifold Instance Mixup (MI-Mix), a contrastive objective that mixes the hidden representations of DNA sequences at randomly selected layers and trains the model to recognize and differentiate these mixed proportions at the output layer. We further enhance it with the proposed Curriculum Contrastive Learning (C$^2$LR) strategy. Empirical results on 23 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios. For example, it identifies twice more species from a mixture of unlabeled genomic sequences, doubles the Adjusted Rand Index (ARI) in species clustering, and outperforms the top baseline's performance in 10-shot species classification with just a 2-shot training. Model, codes, and data are publicly available at \url{https://github.com/MAGICS-LAB/DNABERT_S}.

翻译：我们提出了DNABERT-S，一种定制的基因组模型，该模型通过开发物种感知嵌入，在嵌入空间中自然地聚类和分离不同物种的DNA序列。从基因组序列（即DNA和RNA）中区分物种至关重要但极具挑战性，因为许多现实世界中的物种尚未被表征，缺乏已知的基因组作为参考。因此，基于嵌入的方法被用于以无监督的方式区分物种。DNABERT-S建立在名为DNABERT-2的预训练基因组基础模型之上。为了促进对易出错的长读长DNA序列生成有效的嵌入，我们引入了流形实例混合（MI-Mix），这是一种对比学习目标，它在随机选择的层中混合DNA序列的隐藏表示，并训练模型在输出层识别和区分这些混合比例。我们进一步通过提出的课程对比学习（C$^2$LR）策略对其进行增强。在23个多样化数据集上的实证结果表明了DNABERT-S的有效性，尤其是在现实世界中标签稀缺的场景下。例如，它从未标记的基因组序列混合物中识别出的物种数量翻倍，在物种聚类中将调整兰德指数（ARI）提高了一倍，并且在仅使用2个样本进行训练的情况下，在10样本物种分类任务中超越了顶级基线的性能。模型、代码和数据已在\url{https://github.com/MAGICS-LAB/DNABERT_S}公开提供。