Radiologists must utilize multiple modal images for tumor segmentation and diagnosis due to the limitations of medical imaging and the diversity of tumor signals. This leads to the development of multimodal learning in segmentation. However, the redundancy among modalities creates challenges for existing subtraction-based joint learning methods, such as misjudging the importance of modalities, ignoring specific modal information, and increasing cognitive load. These thorny issues ultimately decrease segmentation accuracy and increase the risk of overfitting. This paper presents the complementary information mutual learning (CIML) framework, which can mathematically model and address the negative impact of inter-modal redundant information. CIML adopts the idea of addition and removes inter-modal redundant information through inductive bias-driven task decomposition and message passing-based redundancy filtering. CIML first decomposes the multimodal segmentation task into multiple subtasks based on expert prior knowledge, minimizing the information dependence between modalities. Furthermore, CIML introduces a scheme in which each modality can extract information from other modalities additively through message passing. To achieve non-redundancy of extracted information, the redundant filtering is transformed into complementary information learning inspired by the variational information bottleneck. The complementary information learning procedure can be efficiently solved by variational inference and cross-modal spatial attention. Numerical results from the verification task and standard benchmarks indicate that CIML efficiently removes redundant information between modalities, outperforming SOTA methods regarding validation accuracy and segmentation effect.

翻译：由于医学成像的局限性和肿瘤信号的多样性，放射科医生必须利用多模态图像进行肿瘤分割与诊断。这推动了分割任务中多模态学习的发展。然而，模态间的冗余性给现有基于减法的联合学习方法带来了挑战，例如错误判断模态重要性、忽略特定模态信息以及增加认知负荷。这些棘手问题最终会降低分割精度并增加过拟合风险。本文提出互补信息互学习框架，能够通过数学模型表征并解决模态间冗余信息的负面影响。CIML采用加法思想，通过归纳偏置驱动的任务分解和基于消息传递的冗余过滤来消除模态间冗余信息。CIML首先基于专家先验知识将多模态分割任务分解为多个子任务，最小化模态间的信息依赖性。此外，CIML引入了一种方案，使每个模态能够通过消息传递以加法方式从其他模态提取信息。为实现所提取信息的非冗余性，冗余过滤被转化为受变分信息瓶颈启发的互补信息学习过程。该互补信息学习流程可通过变分推断和跨模态空间注意力机制高效求解。验证任务和标准基准的数值结果表明，CIML能有效消除模态间冗余信息，在验证精度和分割效果方面均优于当前最先进方法。