Effective DNA embedding remains crucial in genomic analysis, particularly in scenarios lacking labeled data for model fine-tuning, despite the significant advancements in genome foundation models. A prime example is metagenomics binning, a critical process in microbiome research that aims to group DNA sequences by their species from a complex mixture of DNA sequences derived from potentially thousands of distinct, often uncharacterized species. To fill the lack of effective DNA embedding models, we introduce DNABERT-S, a genome foundation model that specializes in creating species-aware DNA embeddings. To encourage effective embeddings to error-prone long-read DNA sequences, we introduce Manifold Instance Mixup (MI-Mix), a contrastive objective that mixes the hidden representations of DNA sequences at randomly selected layers and trains the model to recognize and differentiate these mixed proportions at the output layer. We further enhance it with the proposed Curriculum Contrastive Learning (C$^2$LR) strategy. Empirical results on 18 diverse datasets showed DNABERT-S's remarkable performance. It outperforms the top baseline's performance in 10-shot species classification with just a 2-shot training while doubling the Adjusted Rand Index (ARI) in species clustering and substantially increasing the number of correctly identified species in metagenomics binning. The code, data, and pre-trained model are publicly available at https://github.com/Zhihan1996/DNABERT_S.

翻译：尽管基因组基础模型取得了显著进展，但在缺乏标注数据以进行微调的场景中，有效的DNA嵌入仍是基因组分析的关键。一个典型例子是宏基因组学分箱——这一微生物组研究中的关键过程，旨在从可能来自数千种不同（且通常是未知）物种的复杂DNA序列混合物中，将序列按物种进行分组。为填补当前缺乏有效DNA嵌入模型的空白，我们提出了DNABERT-S，这是一个专门创建物种感知DNA嵌入的基因组基础模型。为了在易出错的长读段DNA序列中生成有效的嵌入，我们引入了流形实例混合（MI-Mix），这是一种对比学习目标：在随机选择的层中混合DNA序列的隐藏表示，并训练模型在输出层识别和区分这些混合比例。我们进一步通过提出的课程对比学习（C$^2$LR）策略对其进行增强。在18个不同数据集上的实验结果表明，DNABERT-S性能卓越：在10样本物种分类任务中，仅需2样本训练即可超越最佳基线的表现；在物种聚类任务中，其调整兰德指数（ARI）翻倍；在宏基因组学分箱中，正确识别的物种数量显著增加。代码、数据和预训练模型已在https://github.com/Zhihan1996/DNABERT_S 公开提供。