Deep learning has enabled the development of highly robust foundation models for various pathological tasks across diverse diseases and patient cohorts. Among these models, vision-language pre-training, which leverages large-scale paired data to align pathology image and text embedding spaces, and provides a novel zero-shot paradigm for downstream tasks. However, existing models have been primarily data-driven and lack the incorporation of domain-specific knowledge, which limits their performance in cancer diagnosis, especially for rare tumor subtypes. To address this limitation, we establish a Knowledge-enhanced Pathology (KEEP) foundation model that harnesses disease knowledge to facilitate vision-language pre-training. Specifically, we first construct a disease knowledge graph (KG) that covers 11,454 human diseases with 139,143 disease attributes, including synonyms, definitions, and hypernym relations. We then systematically reorganize the millions of publicly available noisy pathology image-text pairs, into 143K well-structured semantic groups linked through the hierarchical relations of the disease KG. To derive more nuanced image and text representations, we propose a novel knowledge-enhanced vision-language pre-training approach that integrates disease knowledge into the alignment within hierarchical semantic groups instead of unstructured image-text pairs. Validated on 18 diverse benchmarks with more than 14,000 whole slide images (WSIs), KEEP achieves state-of-the-art performance in zero-shot cancer diagnostic tasks. Notably, for cancer detection, KEEP demonstrates an average sensitivity of 89.8% at a specificity of 95.0% across 7 cancer types. For cancer subtyping, KEEP achieves a median balanced accuracy of 0.456 in subtyping 30 rare brain cancers, indicating strong generalizability for diagnosing rare tumors.

翻译：深度学习已推动开发出适用于多种疾病和患者群体的高度鲁棒性病理学基础模型。在这些模型中，视觉-语言预训练通过利用大规模配对数据来对齐病理图像与文本的嵌入空间，为下游任务提供了一种新颖的零样本范式。然而，现有模型主要依赖数据驱动，缺乏领域特定知识的融入，这限制了其在癌症诊断中的性能，特别是对于罕见肿瘤亚型。为应对这一局限，我们构建了一个知识增强的病理学（KEEP）基础模型，该模型利用疾病知识来促进视觉-语言预训练。具体而言，我们首先构建了一个覆盖11,454种人类疾病、包含139,143个疾病属性（如同义词、定义和上位关系）的疾病知识图谱（KG）。随后，我们系统性地重组了数百万公开可用的噪声病理图像-文本对，将其整理为143K个通过疾病知识图谱的层次关系链接的结构化语义组。为获得更精细的图像和文本表示，我们提出了一种新颖的知识增强视觉-语言预训练方法，该方法将疾病知识整合到层次化语义组内的对齐中，而非非结构化的图像-文本对中。在包含超过14,000张全切片图像（WSIs）的18个多样化基准测试中验证，KEEP在零样本癌症诊断任务中实现了最先进的性能。值得注意的是，在癌症检测方面，KEEP在7种癌症类型中实现了平均灵敏度为89.8%（特异性为95.0%）。在癌症亚型分型方面，KEEP在对30种罕见脑癌进行亚型分型时获得了0.456的中位平衡准确率，显示出其在诊断罕见肿瘤方面强大的泛化能力。