Transcriptome foundation models TFMs hold great promises of deciphering the transcriptomic language that dictate diverse cell functions by self-supervised learning on large-scale single-cell gene expression data, and ultimately unraveling the complex mechanisms of human diseases. However, current TFMs treat cells as independent samples and ignore the taxonomic relationships between cell types, which are available in cell ontology graphs. We argue that effectively leveraging this ontology information during the TFM pre-training can improve learning biologically meaningful gene co-expression patterns while preserving TFM as a general purpose foundation model for downstream zero-shot and fine-tuning tasks. To this end, we present single cell, Cell-ontology guided TFM scCello. We introduce cell-type coherence loss and ontology alignment loss, which are minimized along with the masked gene expression prediction loss during the pre-training. The novel loss component guide scCello to learn the cell-type-specific representation and the structural relation between cell types from the cell ontology graph, respectively. We pre-trained scCello on 22 million cells from CellxGene database leveraging their cell-type labels mapped to the cell ontology graph from Open Biological and Biomedical Ontology Foundry. Our TFM demonstrates competitive generalization and transferability performance over the existing TFMs on biologically important tasks including identifying novel cell types of unseen cells, prediction of cell-type-specific marker genes, and cancer drug responses.

翻译：转录组基础模型（TFMs）通过在大规模单细胞基因表达数据上进行自监督学习，有望破译决定多样化细胞功能的转录组语言，并最终揭示人类疾病的复杂机制。然而，当前的TFMs将细胞视为独立样本，忽略了细胞类型间的分类学关系，而这些关系在细胞本体论图中是可用的。我们认为，在TFM预训练过程中有效利用这种本体论信息，可以改进对具有生物学意义的基因共表达模式的学习，同时保持TFM作为下游零样本和微调任务的通用基础模型。为此，我们提出了单细胞、细胞本体论指导的TFM——scCello。我们引入了细胞类型一致性损失和本体论对齐损失，这两者在预训练过程中与掩码基因表达预测损失一同被最小化。这些新颖的损失组件分别指导scCello从细胞本体论图中学习细胞类型特异性表示以及细胞类型间的结构关系。我们利用来自CellxGene数据库的2200万个细胞预训练了scCello，这些细胞的细胞类型标签已映射到开放生物与生物医学本体论铸造厂（OBO Foundry）的细胞本体论图。我们的TFM在识别未见细胞的新的细胞类型、预测细胞类型特异性标记基因以及癌症药物反应等具有重要生物学意义的任务上，相较于现有TFMs，展现出了具有竞争力的泛化性和可迁移性能。