Effective DNA embedding remains crucial in genomic analysis, particularly in scenarios lacking labeled data for model fine-tuning, despite the significant advancements in genome foundation models. A prime example is metagenomics binning, a critical process in microbiome research that aims to group DNA sequences by their species from a complex mixture of DNA sequences derived from potentially thousands of distinct, often uncharacterized species. To fill the lack of effective DNA embedding models, we introduce DNABERT-S, a genome foundation model that specializes in creating species-aware DNA embeddings. To encourage effective embeddings to error-prone long-read DNA sequences, we introduce Manifold Instance Mixup (MI-Mix), a contrastive objective that mixes the hidden representations of DNA sequences at randomly selected layers and trains the model to recognize and differentiate these mixed proportions at the output layer. We further enhance it with the proposed Curriculum Contrastive Learning (C$^2$LR) strategy. Empirical results on 18 diverse datasets showed DNABERT-S's remarkable performance. It outperforms the top baseline's performance in 10-shot species classification with just a 2-shot training while doubling the Adjusted Rand Index (ARI) in species clustering and substantially increasing the number of correctly identified species in metagenomics binning. The code, data, and pre-trained model are publicly available at https://github.com/Zhihan1996/DNABERT_S.

翻译：尽管基因组基础模型取得了显著进展，但在缺乏标记数据用于模型微调的场景中，有效的DNA嵌入仍是基因组分析的关键。宏基因组学分箱是微生物组研究中的关键流程，其目标是从可能包含数千种不同且通常未表征物种的复杂DNA序列混合物中，按物种对DNA序列进行分组。针对当前缺乏有效DNA嵌入模型的问题，我们提出DNABERT-S——一种专门生成物种感知DNA嵌入的基因组基础模型。为促使模型对易出错的超长读长DNA序列生成有效嵌入，我们引入了流形实例混合（MI-Mix）对比学习目标：在随机层对DNA序列的隐层表示进行混合，并训练模型在输出层识别与区分这些混合比例。我们进一步通过提出的课程对比学习（C$^2$LR）策略增强其性能。在18个不同数据集上的实验结果表明，DNABERT-S具有卓越性能：在10样本物种分类中，仅需2样本训练即可超越最优基线的性能；在物种聚类任务中，调整兰德指数（ARI）实现翻倍；在宏基因组学分箱中显著提升了正确识别的物种数量。相关代码、数据及预训练模型已公开于https://github.com/Zhihan1996/DNABERT_S。