Multimodal learning significantly benefits cancer survival prediction, especially the integration of pathological images and genomic data. Despite advantages of multimodal learning for cancer survival prediction, massive redundancy in multimodal data prevents it from extracting discriminative and compact information: (1) An extensive amount of intra-modal task-unrelated information blurs discriminability, especially for gigapixel whole slide images (WSIs) with many patches in pathology and thousands of pathways in genomic data, leading to an ``intra-modal redundancy" issue. (2) Duplicated information among modalities dominates the representation of multimodal data, which makes modality-specific information prone to being ignored, resulting in an ``inter-modal redundancy" issue. To address these, we propose a new framework, Prototypical Information Bottlenecking and Disentangling (PIBD), consisting of Prototypical Information Bottleneck (PIB) module for intra-modal redundancy and Prototypical Information Disentanglement (PID) module for inter-modal redundancy. Specifically, a variant of information bottleneck, PIB, is proposed to model prototypes approximating a bunch of instances for different risk levels, which can be used for selection of discriminative instances within modality. PID module decouples entangled multimodal data into compact distinct components: modality-common and modality-specific knowledge, under the guidance of the joint prototypical distribution. Extensive experiments on five cancer benchmark datasets demonstrated our superiority over other methods.

翻译：多模态学习显著提升了癌症生存预测的性能，尤其是病理图像与基因组数据的整合。尽管多模态学习在癌症生存预测中具有优势，但多模态数据中的大量冗余阻碍了模型提取判别性且紧凑的信息：(1) 模态内部大量与任务无关的信息模糊了判别性，特别是针对包含众多病理图像块的全切片图像（WSIs）和基因组数据中数千条通路，导致"模态内冗余"问题；(2) 模态间的重复信息主导了多模态数据的表征，使得模态特异性信息容易被忽略，引发"模态间冗余"问题。为解决这些问题，我们提出新型框架——原型信息瓶颈与解耦（PIBD），包含用于处理模态内冗余的原型信息瓶颈（PIB）模块和用于处理模态间冗余的原型信息解耦（PID）模块。具体而言，PIB提出了一种信息瓶颈变体，通过建模不同风险水平的原型来近似大量实例，从而在模态内筛选判别性实例；PID模块则在联合原型分布指导下，将纠缠的多模态数据解耦为紧凑的独立成分：模态共有知识与模态特有知识。在五个癌症基准数据集上的大量实验证明了本方法相较于其他方法的优越性。