Learning with multiple modalities is crucial for automated brain tumor segmentation from magnetic resonance imaging data. Explicitly optimizing the common information shared among all modalities (e.g., by maximizing the total correlation) has been shown to achieve better feature representations and thus enhance the segmentation performance. However, existing approaches are oblivious to partial common information shared by subsets of the modalities. In this paper, we show that identifying such partial common information can significantly boost the discriminative power of image segmentation models. In particular, we introduce a novel concept of partial common information mask (PCI-mask) to provide a fine-grained characterization of what partial common information is shared by which subsets of the modalities. By solving a masked correlation maximization and simultaneously learning an optimal PCI-mask, we identify the latent microstructure of partial common information and leverage it in a self-attention module to selectively weight different feature representations in multi-modal data. We implement our proposed framework on the standard U-Net. Our experimental results on the Multi-modal Brain Tumor Segmentation Challenge (BraTS) datasets consistently outperform those of state-of-the-art segmentation baselines, with validation Dice similarity coefficients of 0.920, 0.897, 0.837 for the whole tumor, tumor core, and enhancing tumor on BraTS-2020.

翻译：多模态学习对于从磁共振成像数据中自动分割脑肿瘤至关重要。明确优化所有模态共享的公共信息（例如，通过最大化总相关性）已被证明能够获得更好的特征表示，从而提升分割性能。然而，现有方法忽略了模态子集共享的部分公共信息。本文表明，识别这种部分公共信息可以显著增强图像分割模型的判别能力。具体而言，我们引入了一个新颖的概念——部分公共信息掩码（PCI-mask），以细粒度表征哪些模态子集共享何种部分公共信息。通过求解掩码相关最大化并同时学习最优PCI-mask，我们识别出部分公共信息的潜在微结构，并将其应用于自注意力模块中，以选择性加权多模态数据中的不同特征表示。我们在标准U-Net上实现了所提出的框架。在脑肿瘤分割挑战赛（BraTS）数据集上的实验结果表明，我们的方法持续优于最先进的分割基线模型，在BraTS-2020数据集上，全肿瘤、肿瘤核心和增强肿瘤的验证Dice相似系数分别达到0.920、0.897和0.837。