In medical vision, different imaging modalities provide complementary information. However, in practice, not all modalities may be available during inference or even training. Previous approaches, e.g., knowledge distillation or image synthesis, often assume the availability of full modalities for all patients during training; this is unrealistic and impractical due to the variability in data collection across sites. We propose a novel approach to learn enhanced modality-agnostic representations by employing a meta-learning strategy in training, even when only limited full modality samples are available. Meta-learning enhances partial modality representations to full modality representations by meta-training on partial modality data and meta-testing on limited full modality samples. Additionally, we co-supervise this feature enrichment by introducing an auxiliary adversarial learning branch. More specifically, a missing modality detector is used as a discriminator to mimic the full modality setting. Our segmentation framework significantly outperforms state-of-the-art brain tumor segmentation techniques in missing modality scenarios.

翻译：在医学影像中，不同成像模态提供互补信息。然而实际应用中，推理甚至训练阶段可能无法获取所有模态。以往方法（如知识蒸馏或图像合成）通常假设训练时所有患者均具备完整模态，但由于不同机构数据收集的差异性，这种假设既不现实也不可行。我们提出一种新方法，通过训练阶段采用元学习策略学习增强型模态无关表示，即便仅能获取有限完整模态样本。该方法通过在部分模态数据上元训练、在有限完整模态样本上元测试，将部分模态表示增强为完整模态表示。此外，我们引入辅助对抗学习分支来协同监督特征增强过程：具体而言，利用缺失模态检测器作为判别器模仿完整模态设置。我们的分割框架在缺失模态场景下显著优于现有最先进的脑肿瘤分割技术。