Cancer has relational information residing at varying scales, modalities, and resolutions of the acquired data, such as radiology, pathology, genomics, proteomics, and clinical records. Integrating diverse data types can improve the accuracy and reliability of cancer diagnosis and treatment. There can be disease-related information that is too subtle for humans or existing technological tools to discern visually. Traditional methods typically focus on partial or unimodal information about biological systems at individual scales and fail to encapsulate the complete spectrum of the heterogeneous nature of data. Deep neural networks have facilitated the development of sophisticated multimodal data fusion approaches that can extract and integrate relevant information from multiple sources. Recent deep learning frameworks such as Graph Neural Networks (GNNs) and Transformers have shown remarkable success in multimodal learning. This review article provides an in-depth analysis of the state-of-the-art in GNNs and Transformers for multimodal data fusion in oncology settings, highlighting notable research studies and their findings. We also discuss the foundations of multimodal learning, inherent challenges, and opportunities for integrative learning in oncology. By examining the current state and potential future developments of multimodal data integration in oncology, we aim to demonstrate the promising role that multimodal neural networks can play in cancer prevention, early detection, and treatment through informed oncology practices in personalized settings.

翻译：癌症相关信息存在于不同尺度、模态和分辨率的数据中，例如影像学、病理学、基因组学、蛋白质组学及临床记录。整合多类型数据可提升癌症诊疗的准确性与可靠性。某些疾病相关信息过于细微，人类或现有技术工具难以通过视觉识别。传统方法通常聚焦于生物系统在单一尺度上的局部或单模态信息，无法全面覆盖数据的异质性特征。深度神经网络推动了复杂多模态数据融合方法的发展，使其能够从多源数据中提取并整合关联信息。近期如图神经网络（GNNs）与Transformer等深度学习框架在多模态学习中展现出显著成效。本文深入分析了GNNs与Transformer在肿瘤学场景中实现多模态数据融合的前沿技术，重点梳理了代表性研究成果及其发现。同时探讨了多模态学习的基础理论、固有挑战及肿瘤学整合学习的潜在机遇。通过审视多模态数据整合在肿瘤学中的当前状态与未来发展，旨在阐明多模态神经网络在个性化医疗框架下，通过精准实践推动癌症预防、早期检测与治疗方面的重要前景。