Automatic integration of whole slide images (WSIs) and gene expression profiles has demonstrated substantial potential in precision clinical diagnosis and cancer progression studies. However, most existing studies focus on individual gene sequences and slide level classification tasks, with limited attention to spatial transcriptomics and patch level applications. To address this limitation, we propose a multimodal network, BioMorphNet, which automatically integrates tissue morphological features and spatial gene expression to support tissue classification and differential gene analysis. For considering morphological features, BioMorphNet constructs a graph to model the relationships between target patches and their neighbors, and adjusts the response strength based on morphological and molecular level similarity, to better characterize the tumor microenvironment. In terms of multimodal interactions, BioMorphNet derives clinical pathway features from spatial transcriptomic data based on a predefined pathway database, serving as a bridge between tissue morphology and gene expression. In addition, a novel learnable pathway module is designed to automatically simulate the biological pathway formation process, providing a complementary representation to existing clinical pathways. Compared with the latest morphology gene multimodal methods, BioMorphNet's average classification metrics improve by 2.67%, 5.48%, and 6.29% for prostate cancer, colorectal cancer, and breast cancer datasets, respectively. BioMorphNet not only classifies tissue categories within WSIs accurately to support tumor localization, but also analyzes differential gene expression between tissue categories based on prediction confidence, contributing to the discovery of potential tumor biomarkers.

翻译：全切片图像（WSIs）与基因表达谱的自动整合在精准临床诊断与癌症进展研究中展现出巨大潜力。然而，现有研究大多聚焦于单一基因序列及切片级别的分类任务，对空间转录组学及斑块级别应用的关注有限。为应对此局限，我们提出一种多模态网络BioMorphNet，其能够自动整合组织形态学特征与空间基因表达，以支持组织分类与差异基因分析。在形态特征建模方面，BioMorphNet构建图模型以刻画目标斑块与其邻域斑块间的关系，并依据形态学与分子层面的相似性调整响应强度，从而更精准地表征肿瘤微环境。在多模态交互方面，BioMorphNet基于预定义的通路数据库从空间转录组数据中提取临床通路特征，作为连接组织形态与基因表达的桥梁。此外，我们设计了一种新颖的可学习通路模块，能够自动模拟生物通路的形成过程，为现有临床通路提供互补性表征。相较于最新的形态-基因多模态方法，BioMorphNet在前列腺癌、结直肠癌和乳腺癌数据集上的平均分类指标分别提升了2.67%、5.48%和6.29%。BioMorphNet不仅能够准确分类WSIs内的组织类别以辅助肿瘤定位，还能基于预测置信度分析组织类别间的差异基因表达，有助于发现潜在的肿瘤生物标志物。