Brain lesion segmentation plays an essential role in neurological research and diagnosis. As brain lesions can be caused by various pathological alterations, different types of brain lesions tend to manifest with different characteristics on different imaging modalities. Due to this complexity, brain lesion segmentation methods are often developed in a task-specific manner. A specific segmentation model is developed for a particular lesion type and imaging modality. However, the use of task-specific models requires predetermination of the lesion type and imaging modality, which complicates their deployment in real-world scenarios. In this work, we propose a universal foundation model for 3D brain lesion segmentation, which can automatically segment different types of brain lesions for input data of various imaging modalities. We formulate a novel Mixture of Modality Experts (MoME) framework with multiple expert networks attending to different imaging modalities. A hierarchical gating network combines the expert predictions and fosters expertise collaboration. Furthermore, we introduce a curriculum learning strategy during training to avoid the degeneration of each expert network and preserve their specialization. We evaluated the proposed method on nine brain lesion datasets, encompassing five imaging modalities and eight lesion types. The results show that our model outperforms state-of-the-art universal models and provides promising generalization to unseen datasets.

翻译：脑部病变分割在神经学研究和诊断中起着关键作用。由于脑部病变可能由多种病理改变引起，不同类型的脑部病变通常在不同成像模态上表现出不同的特征。由于这种复杂性，脑部病变分割方法通常以任务特定的方式开发，即针对特定的病变类型和成像模态开发特定的分割模型。然而，使用任务特定模型需要预先确定病变类型和成像模态，这在实际场景中的部署增加了复杂性。在本工作中，我们提出了一种用于3D脑部病变分割的通用基础模型，该模型可以自动分割不同成像模态输入数据中的多种类型脑部病变。我们构建了一种新颖的混合模态专家（MoME）框架，其中多个专家网络分别关注不同的成像模态。一个分层门控网络结合了专家预测并促进了专家协作。此外，我们在训练过程中引入课程学习策略，以避免每个专家网络的退化并保持其专业化。我们在九个脑部病变数据集上评估了所提方法，涵盖了五种成像模态和八种病变类型。结果表明，我们的模型优于最先进的通用模型，并在未见数据集上展现出良好的泛化能力。